{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# 第六章.与学习相关的技巧\n",
    "<p>本章涉及寻找最优权重参数的最优化方法，权重参数的初始值,超参数的设定方法等.</p>"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 6.1.参数的更新\n",
    "<P>神经网络的学习目的是找到使损失函数的值尽可能小的参数,这是寻找最优参数的问题,解决这个问题的过程称为<strong>最优化</strong></P>\n",
    "<P>为了找到最优参数,我们将参数的梯度(偏导数)作为了线索。使用参数的梯度,沿梯度方向更新参数,并重复该步骤多次,从而逐渐靠近该最优参数,这个过程称为随机梯度下降法(SGD)</P>\n",
    "\n",
    "### 6.1.1.探险家的故事(类似于SGD策略)"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 6.1.2.SGD"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "\n",
    "# 实现一个SGD类\n",
    "class SGD:\n",
    "    def __init__(self,lr=0.01):\n",
    "        self.lr = lr\n",
    "\n",
    "    def update(self,params,grads):\n",
    "        for key in params.key():\n",
    "            params[key] -= self.lr*grads[key]"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-10-01T02:13:13.802963700Z",
     "start_time": "2023-10-01T02:13:13.793965800Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 6.1.3.SGD的缺点\n",
    "<P>SGD的缺点是:如果函数的形状非均向,比如呈现延伸状,搜索的路径会非常低效.SGD低效的根本原因:梯度的方向没有指向最小值的方向.因此我们将尝试其他寻找最优参数的方法</P>"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 6.1.4.Momentum(动量)"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "outputs": [],
   "source": [
    "class Momentum:\n",
    "    def __init__(self,lr=0.01,momentum=0.9):\n",
    "        self.lr = lr\n",
    "        self.momentum = momentum\n",
    "        self.v = None\n",
    "\n",
    "    def update(self,params,grads):\n",
    "        # 如果速度为None\n",
    "        if self.v is None:\n",
    "            self.v = {}\n",
    "            for key ,val in params.items():\n",
    "                self.v[key] = np.zeros_like(val)\n",
    "\n",
    "        for key in params.key():\n",
    "            self.v[key] = self.momentum*self.v[key]-self.lr*grads[key]\n",
    "            params[key]+=self.v[key]\n"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-10-01T02:35:33.854284900Z",
     "start_time": "2023-10-01T02:35:33.818229900Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 6.1.5.AdaGrad\n",
    "<p>在学习率的有效技巧中,有一种被称为学习率衰减的方法,随着学习的进行,使学习率逐渐减小。实际上一开始多学,然后逐渐少学的方法,在神经网络中经常被使用</p>"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "outputs": [],
   "source": [
    "class AdaGrad:\n",
    "    def __init__(self,lr=0.01):\n",
    "        self.lr = lr\n",
    "        self.h = None\n",
    "\n",
    "    def update(self,params,grads):\n",
    "        if self.h is None:\n",
    "            self.h = {}\n",
    "            for key,val in params.items():\n",
    "                self.h[key] = np.zeros_like(val)\n",
    "\n",
    "        for key in params.keys():\n",
    "            self.h[key]+=grads[key]*grads[key]\n",
    "            params[key]-=self.lr*grads[key]/(np.sqrt(self.h[key])+1e-7)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-10-01T03:01:41.365702300Z",
     "start_time": "2023-10-01T03:01:41.340660Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 6.1.6.Adam"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 6.2.权重的初始值\n",
    "<p>在神经网络的学习中,神经网络权重的初始值设定是非常重要的.它往往能决定神经网络的学习是否成功</p>"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 6.2.1.可以将权重的初始值设为0吗?\n",
    "<p>不能,因为会导致权重的均一化,使得神经网络拥有许多不同的权重失去了意义.</p>"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 6.2.2.隐藏层的激活函数分布:\n",
    "<P>观察权重初始值是如何影响隐藏层的激活值的分布</P>"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "outputs": [
    {
     "data": {
      "text/plain": "<Figure size 432x288 with 5 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEICAYAAABfz4NwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dfXSV9Z3v/ffXQKEzwIhU6IbABEmO8mhGYoA13r21MWOwHNA6ItoDcYnSFfXYEbt60sPtXXqfY41nHce2t5RbalwFZ45UZkbwKMYi6OoalzQDNSrE0YAwQ0IGkIcBWkghfu8/9rXDTvZOsvOwsx/yea21Fzu/ff12rv3lyvX9PVz7d5m7IyIiclmqd0BERNKDEoKIiABKCCIiElBCEBERQAlBREQCSggiIgIoIbQxs4NmdnOq9yPdKC6xFJNYikl8mRaXrE4IZvawme0ysxYz+0Wq9ycdmNkwM6s2s38xszNm9r6ZzU/1fqWamf2NmTWb2Wkz+9TM7k/1PqULMysws/Nm9jep3pd0YGbvBPE4Gzw+SfU+9ZesTgjAYeC/Ay+kekfiMbMhKfi1Q4BDwP8J/AnwOPCymeWlYF/iSlFcngTy3H0UsBD472Y2OwX7EVeKYhKxBvinFP7+uFIck4fdfUTwuDqF+xGjL3HJ6oTg7v/g7puB4z2pZ2bFZvaemZ0KWo3PmtmXgtfWmNnTHbb/32b2V8Hz8Wb292Z2zMwOmNkjUdutNrO/C1qjp4F7+/whe8jdf+fuq939oLt/4e6vAQeAbk9+WR6Xve7eEvkxeEzprl42xyTYjyXAKWB7D+pkdUx6KyPi4u5Z/yDcS/hFN9scBG4Ons8G5hJuTecBHwN/FbxWTLjncVnw81eA3wPjCCfY3cD/DXwJuAr4DLgl2HY1cAG4Ldj2y2kQm3HAeeCawR4X4GfBPjvwW2DEYI4JMAr4FJgY7M/f6O/HAd4BjgGfA+8CN2ZLXLK6h9Bb7r7b3Xe6+0V3Pwg8R3iIBXevBf4dKAk2XwK84+5HgOuBK939/3H3P7j7Z8DPg20i3nP3zR5unZ8bqM8Uj5kNBf4WWO/u/9zd9tkeF3d/EBgJ/B/APwAtXdfI+pj8N6Da3Q/1pFKWxwTgvxA+KU8A1gH/28y67U1mQlwGZUIwszeiJoS+Fef1/2Bmr5nZvwVdsB8RztgR64H/FDz/T8CLwfM/BcYHXcJTZnYK+K+Es3xEj/64ksXMLiO8338AHg7KBn1c3L3V3f8RyAUqBmtMzKwQuBl4Js5rgzImEe7+G3c/4+4t7r6ecC/h1myISyonZVLG3bu7qmYt8D5wt7ufCcbx/jLq9b8B9pjZtcBUYHNQfgg44O4FXf36Xu52vzEzA6oJH1C3uvsFUFw6GAJMGcQxuZHwsMa/hg8XRgA5ZjbN3a/rpm62xqQzDlg2HCtZ3UMwsyFmNhzIIXwwD7fEZuBHAqeBs2Z2DVAR/aK7NxK+6uJF4O+jumi1wGkz+y9m9mUzyzGzGWZ2fb99qP6xlvAB9x972L3MyriY2VgzW2JmI4J9uwW4G9iRQPWsjAnhoZApQGHw+P+A14FbEqibrTHBzC43s1si55KgJ/A14M0Eqqd9XLI6IQD/F3AOqCTcBTsXlHXnu8A9wBnCY3W/jLPNemAml7p1uHsr8B8J/wEdIDzp9DzhyzvTgpn9KfBtwvv4b111cePI1rg44T/ORuAk8D8JT/ZtSaBuVsbE3X/v7v8WeQBngfPufiyB6lkZk8BQwhepRCaV/zNwm7sn8l2EtI+LBbPU0kNm9jXCXbw8d/8i1fuTLhSXWIpJLMUkvlTHJdt7CEkRXJ3zHeB5HcyXKC6xFJNYikl86RAXJYQeMrOphL+oEwJ+nOLdSRuKSyzFJJZiEl+6xEVDRiIiAqiHICIigYz9HsJXvvIVz8vLS/VuJN3u3bs/d/crE9lWMYlvMMRFMYlPfz+xuopJxiaEvLw8du3alerdSDoz+5dEt1VM4hsMcVFM4tPfT6yuYqIhIxERAZQQREQkoIQgIiKAEoKIiASUEEREBFBCEBGRgBKCiIgASggiIhJQQhARESBLEkJe5eup3gVJov76/82rfF3HShyKSXbryf9vViSEbKETloikkhKCiIgASggiIhJQQpB+cf78eYqLi7n22muZPn06P/jBDwA4ceIEpaWlFBQUUFpaysmTJ9vqPPnkk+Tn53P11Vfz5ptvtpXv3r2bmTNnkp+fzyOPPELkJk4tLS3cddddADPM7Ddmljdwn1Ak+ykhSL8YNmwYO3bs4IMPPqCuro6amhp27txJVVUVJSUlNDQ0UFJSQlVVFQD19fVs3LiRvXv3UlNTw4MPPkhraysAFRUVrFu3joaGBhoaGjj/2W4AqqurGT16NMAe4BngqZR8WJEs1W1CMLPhZlZrZh+Y2V4z+2FQvtrMmsysLnjcGlXn+2a2z8w+MbNbospnm9lHwWs/NTMLyoeZ2S+DcrX8MpCZMWLECAAuXLjAhQsXMDO2bNlCeXk5AOXl5WzevBmALVu2sGTJEoYNG8bkyZPJz8+ntraW5uZmTp8+zbx58zAzli1bxu8bdrbVibwX8HdASeQYEpG+S6SH0AJ83d2vBQqBMjObG7z2jLsXBo+tAGY2DVgCTAfKgJ+ZWU6w/VpgBVAQPMqC8uXASXfPJwNafskaHjnx1nMZPTzS2tpKYWEhY8eOpbS0lDlz5nDkyBFCoRAAoVCIo0ePAtDU1MTEiRPb6ubm5tLU1ERTUxO5ubntylvPHo+p4+4XgX8HxgzMpxPJft0mBA87G/w4NHh4F1UWARvdvcXdDwD7gGIzCwGj3P09D5/1NgC3RdVZHzxP+5ZfsoZHLpw4nNHDIzk5OdTV1dHY2EhtbS179uzpdNtI4otmZnHLu6pDJ8eima0ws11mtuvYsWPd7ruIJDiHYGY5ZlYHHAW2uftvgpceNrMPzewFMxsdlE0ADkVVbwzKJgTPO5a3q5MJLb9kDY+MmPH1rBgeufzyy7nxxhupqalh3LhxNDc3A9Dc3MzYsWOBcMv/0KFLh0ljYyPjx48nNzeXxsbGduU5I8bE1DGzIcCfACfi7YO7r3P3IncvuvLKhG81LDKoJZQQ3L3V3QuBXMKt/RmEh3+mEB5GagaeDjaPd9LyLsq7qtNOOrX6kjE8kjNyTMYOjxw7doxTp04BcO7cOd566y2uueYaFi5cyPr14c7f+vXrWbRoEQALFy5k48aNtLS0cODAARoaGiguLiYUCjFy5Eh27tyJu7Nhwwb+qGBOW53IewF/CezwrroUaWCgrr4CrtIcnPRVj64ycvdTwDtAmbsfCRLFF8DPgeJgs0ZgYlS1XOBwUJ4bp7xdna5afunU6kuX4ZF0SZLNzc3cdNNNzJo1i+uvv57S0lIWLFhAZWUl27Zto6CggG3btlFZWQnA9OnTWbx4MdOmTaOsrIw1a9aQkxOealq7di33338/+fn5TJkyheFXFQGwfPlyjh8/DjADWAlUpuTD9kAyr76qqakBwsOLwMVMn4NbvXo1EyZMoLCwkMLCQrZu3dpWR5coD4wh3W1gZlcCF9z9lJl9GbgZeMrMQu7eHGx2O+GxboBXgf9lZn8NjCc8eVzr7q1mdiaYkP4NsAz4f6PqlAPvkSEtv4h4wyOhUKhXwyOtZ473eHjE3dcB6wCKiopSFrNZs2bx/vvvx5SPGTOG7du3x62zatUqVq1aFVNeVFTULsG+FiznMXz4cDZt2oSZ7XH34piKaair4cV33nkHCA8v3njjjTz11FOdDi/m5eW1DS8CLFu2jM2bNzN//ny2bNkCcDz4lX8HPGtmlq5/Q5EkOWLECC5cuMANN9zA/PnzAXj00Uf57ne/22776CR5+PBhbr75Zj799FNycnLakuTcuXO59dZbqampYf78+Z3Nwd01wB814yTSQwgBb5vZh8A/EZ5DeA34H8ElpB8CNwGPArj7XuBloB6oAR5y99bgvSqA5wlPNO8H3gjKq4ExZraPDGj5JWt45OyeHRk9PCLxJevqq6amprY6wB+g6+HFdOlNdpYkO9OTS5Sj5+0ydQ4ulbrtIbj7h8CfxSlf2kWdJ4An4pTvItzd71h+Hrizu31JF83NzZSXl9Pa2soXX3zB4sWLWbBgAfPmzWPx4sVUV1czadIkNm3aBLQfHhkyZEjM8Mi9997LuXPnGDp6arvhkaVLl8Kl4ZElKfmw0meR4cVTp05x++2399vwYuT8lujwYrr0JiGcJGfPns2+fft46KGHmDNnDm+88QbPPvssGzZsoKioiKeffprRo0fT1NTE3Llz2+pGkuHQoUO7TJLRc3BmFkmSn0fvh5mtIHwpPJMmTUryp05/3SYEiZWs4ZHolU4zcXhEutafw4uR8kid+vr6L0H3V1+li3hJsqKigscffxwz4/HHH+exxx7jhRdeGDRJMh1o6QqRJErm1VfRdbg0RJRRw4sdk2ROTg6XXXYZDzzwALW1tUDvk2SilyjLJUoIIkmUzKuvIhOxy5cvBxiS6XNwke+rALzyyivMmBEeXe5tktQcXM9pyEgkiZI5vPja97dysOobDB8+HOAzdy/qz31Pls7m4JYuXUpdXR1mRl5eHs899xyQ+Bzc/Pnz2yVJzcH1nBKCiAyozpLkiy++2GmdRC9RjtAcXO9oyEhERAAlBBERCSghiIgIoIQgIiIBJYQslVf5ersvuomIdEcJQUREACUEEREJKCGIiAighCAiIgElBBERAZQQREQkoIQgIiKAEoKIiASUEEREBFBCEBGRQLcJwcyGm1mtmX1gZnvN7IdB+RVmts3MGoJ/R0fV+b6Z7TOzT8zslqjy2Wb2UfDaTy24AaqZDTOzXwblvzGzvP7/qCIi0pVEeggtwNfd/VqgECgzs7mEb9O33d0LgO3Bz5jZNMJ3J5oOlAE/M7Oc4L3WAiuAguBRFpQvB066ez7wDPBUP3w2ERHpgW4TgoedDX4cGjwcWAREblq6HrgteL4I2OjuLe5+ANgHFJtZCBjl7u8F9zbd0KFO5L3+DiiJ9B5ERGRgJDSHYGY5ZlYHHAW2uftvgHHu3gwQ/Ds22HwCcCiqemNQNiF43rG8XR13vwj8OzAmzn6sMLNdZrbr2LFjiX1CGRCHDh3ipptuYurUqUyfPp2f/OQnAKxevZoJEyZQWFhIYWEhW7dubavz5JNPkp+fz9VXX82bb77ZVr57925mzpxJfn4+jzzyCJF7o7e0tHDXXXcBzNDQYuY6f/48xcXFXHvttUyfPp0f/OAHAJw4cYLS0lIKCgooLS3l5MmTbXV0rAyMhBKCu7e6eyGQS7i1P6OLzeO17L2L8q7qdNyPde5e5O5FV155ZXe7LQNoyJAhPP3003z88cfs3LmTNWvWUF9fD8Cjjz5KXV0ddXV13HrrrQDU19ezceNG9u7dS01NDQ8++CCtra0AVFRUsG7dOhoaGmhoaOD8Z7sBqK6uZvTo0QB70NBixho2bBg7duzggw8+oK6ujpqaGnbu3ElVVRUlJSU0NDRQUlJCVVUV0LNjpaamBtCx0ls9usrI3U8B7xAe+z8SDAMR/Hs02KwRmBhVLRc4HJTnxilvV8fMhgB/Apzoyb4NpGS1hk+89VzGtnBCoRDXXXcdACNHjmTq1Kk0NTV1uv2WLVtYsmQJw4YNY/LkyeTn51NbW0tzczOnT59m3rx5mBnLli3j9w072+qUl5dH3kJDixnKzBgxYgQAFy5c4MKFC5hZu//f8vJyNm/eDPTsWImuo2Ol5xK5yuhKM7s8eP5l4Gbgn4FXgUjEy4EtwfNXgSXBlUOTCU8e1wbDSmfMbG7wH7OsQ53Ie/0lsMMjZ8Y0lKzW8IUTh7OiNXzw4EHef/995syZA8Czzz7LrFmzuO+++9qGAZqampg48VK7ITc3l6amJpqamsjNzW1X3nr2eEydroYWQcOL6a61tZXCwkLGjh1LaWkpc+bM4ciRI4RCISDcwDh6NNzGTPRYefS1f+XF7e/H1NEwdOIS6SGEgLfN7EPgnwjPIbwGVAGlZtYAlAY/4+57gZeBeqAGeMjdW4P3qgCeJzzRvB94IyivBsaY2T5gJcEVS+kqWa3hETO+nvGt4bNnz3LHHXfw4x//mFGjRlFRUcH+/fupq6sjFArx2GOPARAv35tZ3PKITl6LX5gmw4sD1ZsErsqky7ZzcnKoq6ujsbGR2tpa9uzZ0+m2yTxW0uU4SRdDutvA3T8E/ixO+XGgpJM6TwBPxCnfBcTMP7j7eeDOBPY37US3ht99912effZZNmzYQFFREU8//TSjR4+mqamJuXPnttWJtHCGDh3aroWTM3JMp61hM4u0cD6P/v1mtoLwpbxMmjQpyZ+2axcuXOCOO+7gW9/6Ft/85jcBGDduXNvrDzzwAAsWLADCMTh06NK1B42NjYwfP57c3FwaGxvbleeMGBNTJxOGFuFSb/K6667jzJkzzJ49m9LSUiDcm/zud7/bbvvo3uThw4e5+eab+fTTT8nJyWnrTc6dO5c/mnJ90JtcQHV1NcBFd883syWEe5N3DfBH7ZXLL7+cG2+8kZqaGsaNG0dzczOhUIjm5mbGjg1fp5LosdJ65nhGHyvpQN9U7oN0aA2nSwvH3Vm+fDlTp05l5cqVbeXNzc1tz1955RVmzAi3BxYuXMjGjRtpaWnhwIEDNDQ0UFxcTCgUYuTIkezcuRN3Z8OGDfxRwZy2OuvXR65OTv+hRRi43iRwPHiLtO9NHjt2jFOnTgFw7tw53nrrLa655pp2/7/r169n0aJFQOLHytk9OzL6WEkH3fYQJL5ktIYzuYXz7rvv8uKLLzJz5kwKCwsB+NGPfsRLL71EXV0dZkZeXh7PPfccANOnT2fx4sVMmzaNIUOGsGbNGnJywt9fXLt2Lffeey/nzp1j/vz51P9xEQDLly9n6dKlEO5lriT8BciMkczeJPAHyIzeZHNzM+Xl5bS2tvLFF1+wePFiFixYwLx581i8eDHV1dVMmjSJTZs2AYkfK0NHT2X4VdlxrKSKEkIvdNUajkyKdWwN33PPPaxcuZLDhw+3tXBycnLaWjhz5szh7J4djJq9oK1OJrVwbrjhhri9msjEejyrVq1i1apVMeVFRUXtxpRfq3wdgOHDh7Np0ybMbI+7F/fDbg+YeL3Jxx9/HDPj8ccf57HHHuOFF15Iem8SWAdQVFSUsmNp1qxZvP/++zHlY8aMYfv27XHrJHKs5AXHCWT2sZJKSgi9kKzWsFo42WkgepP19fVfgszoTUr6UkLohWS1htXCyT4D1Zv81a9+FbmkMu17k5K+lBBEkmigepMPP/zwkOCy7ROoNym9pIQgkkQD1ZsEPnP3oj7vsAxquuxUREQAJQQREQkoIYiICKCEICIiASUEEREBlBBERCSghCAiIoASgoiIBJQQREQEUEIQEZGAEoKIiABKCCIiEug2IZjZRDN728w+NrO9ZvadoHy1mTWZWV3wuDWqzveDG35/Yma3RJXPNrOPgtd+GrnNn5kNM7NfZtJNwkVEsk0iPYSLwGPuPhWYCzxkZtOC155x98LgsRUgeG0JMB0oA35mZjnB9msJ38KvIHiUBeXLgZPung88Q/gm4SIiMoC6TQju3uzuvw2enwE+BiZ0UWURsNHdW9z9ALAPKDazEDDK3d8Lbt6xAbgtqk7kfpFpf5NwEZFs1KM5hGAo58+A3wRFD5vZh2b2gpmNDsomAIeiqjUGZROC5x3L29Vx94tA5CbhHX//CjPbZWa7jh071pNdF5E0cejQIW666SamTp3K9OnT+clPfgLA6tWrmTBhAoWFhRQWFrJ169a2Ok8++ST5+flcffXVvPnmm23lu3fvZubMmeTn53Pirefa7j3R0tLCXXfdBTBDw9CJSzghmNkI4O+Bv3L304SHf6YAhUAz8HRk0zjVvYvyruq0L3Bf5+5F7l505ZVXJrrrIpJGhgwZwtNPP83HH3/Mzp07WbNmDfX19QA8+uij1NXVUVdX13YTofr6ejZu3MjevXupqanhwQcfpLW1FYCKigrWrVtHQ0MDF04c5vxnuwGorq5m9OjRAHvQMHTCEkoIZjaUcDL4W3f/BwB3P+Lure7+BfBzIHLf30ZgYlT1XOBwUJ4bp7xdHd0kXCS7hUIhrrvuOgBGjhzJ1KlTaWpq6nT7LVu2sGTJEoYNG8bkyZPJz8+ntraW5uZmTp8+zbx58zAzRsz4Or9v2NlWp7y8PPIWGoZOUCJXGRlQDXzs7n8dVR6K2ux2wpkY4FVgSXDl0GTCk8e17t4MnDGzucF7LgO2RNWJ/O/pJuEig8TBgwd5//33mTNnDgDPPvsss2bN4r777uPkyZMANDU1MXHipTZmbm4uTU1NNDU1kZt7qY2ZM3IMrWePx9TRMHTiEukh/DmwFPh6h0tM/0dwCemHwE3AowDuvhd4GagHaoCH3L01eK8K4HnCE837gTeC8mpgTHCT8JVAZb98OhFJW2fPnuWOO+7gxz/+MaNGjaKiooL9+/dTV1dHKBTiscceA4h7T2ozi1se0clrGobuRiJXGf2ju5u7z4q+xNTdl7r7zKB8YdADiNR5wt2nuPvV7v5GVPkud58RvPZwpBfg7ufd/U53z3f3Ynf/LDkfV5Kls4nCEydOUFpaSkFBAaWlpW2tPkhsovCRRx7RRGEWunDhAnfccQff+ta3+OY3vwnAuHHjyMnJ4bLLLuOBBx6gtrYWCPcIDh26dJ1KY2Mj48ePJzc3l8bGS9eptJ45Ts6IMTF1NAydOH1TWfpFZxOFVVVVlJSU0NDQQElJCVVVVUDiE4UNDQ2aKMwy7s7y5cuZOnUqK1eubCtvbm5rU/LKK68wY8YMABYuXMjGjRtpaWnhwIEDNDQ0UFxcTCgUYuTIkezcuRN35+yeHfxRwZy2OuvXR65k1zB0opQQeiFZreFMvmyus4nC6Mm98vJyNm/eDCQ+Ubhs2TJNFGaZd999lxdffJEdO3a0u8T0e9/7HjNnzmTWrFm8/fbbPPPMMwBMnz6dxYsXM23aNMrKylizZg05OeHvuq5du5b777+f/Px8ho7+KsOvKgJg+fLlHD9+HGAGGoZOmBJCLySrNZwtl81FTxQeOXKEUCh8/UEoFOLo0aNA4hOFubm5PZ4ohPSZLByoxgNwVaYs/XLDDTfg7nz44YftLjF98cUX+eijj/jwww959dVX244bgFWrVrF//34++eQT5s+f31ZeVFTEnj172L9/P1eUVhBpHwwfPpxNmzYB7NEwdOKUEHohWa3hbLhsruNEYWeSOVEYbJsWk4UD1XgALmrpF+krJYQ+6s/WcKZfNtfZRGFkbLi5uZmxY8cCiU8UNjY2ZvRE4UA1HoDjwa/MmMaDpB8lhD5Ih9ZwurSEO5sojJ7cW79+PYsWLWorT2SicMOGDVkzUZjMxgPwB8icxoOkJyWEXkpGaziTL5vrbKKwsrKSbdu2UVBQwLZt26isDM/tJTpROGXKlKyYKFTjQTLBkFTvQCbqrjVcWVkZ0xq+5557WLlyJYcPH25rDefk5LS1hufMmcPZPTsYNXtBu/cKpH1rODJRGM/27dvjlq9atYpVq1bFlEcmCiNeq3wduDRRaGZ73L04pmKa6qrxEAqF+qXxUF9f/yXIjMaDpC/1EHohWa1hXTaXfZI1lNbxmnsuDRGlfeNB0pd6CL2QrNZwXtAShsxtDUt7kcbDzJkzKSwsBOBHP/oRlZWVLF68mOrqaiZNmhS5RLJd42HIkCExjYd7772Xc+fOMXT01HaNh4cffnhIsPTLCcI3qBLpMSUEkSQaqMYD8Jm7F/V5hwW4FN+DVd9I8Z4MLA0ZiYgIoIQgIiIBJQQREQGUEEREJKCEICIigBKCiIgElBBERARQQhARkUC3CcHMJprZ22b2sZntNbPvBOVXmNk2M2sI/h0dVef7wc06PjGzW6LKZ5vZR8FrP40s0Wtmw8zsl5lygw8RkWyUSA/hIvCYu08F5gIPmdk0wmvrbHf3AmB78DPBa0uA6UAZ8DMzywneay2wAigIHmVB+XLgpG7wISKSOt0mBHdvdvffBs/PAB8DE4BFQGQ5zvXAbcHzRcBGd29x9wPAPqDYzELAKHd/L1h4a0OHOpH30g0+RLKY7kmevno0hxAE9c+A3wDj3L0ZwkkDGBtsNgE4FFWtMSibEDzvWN6ujm7wIZLddE/y9JVwQjCzEcDfA3/l7qe72jROmXdR3lWd9gW6wYdIxtM9ydNXQgnBzIYSTgZ/6+7/EBQfCYaBCP49GpQ3AhOjqucCh4Py3Djl7eroBh8ig4fuSZ5eErnKyIBq4GN3/+uol14FIim4HNgSVb4kuHJoMuHJ49pgWOmMmc0N3nNZhzqR99INPkQGAd1WNP0kcj+EPweWAh+ZWV1Q9l+BKuBlM1sO/CtwJ4C77zWzl4F6wlcoPeTurUG9CuAXwJeBN4IHhBPOi7rBh8jgMBC3Fc2ke5Kni0SuMvpHdzd3n+XuhcFjq7sfd/cSdy8I/j0RVecJd5/i7le7+xtR5bvcfUbw2sORXoC7n3f3O909392L3f2z5HxcEUm1gbqtaCbdkzxd6I5pIjKgBuq2okuXLoVL9yTXqEMClBBEZEDpnuTpS2sZiYgIoIQgIiIBJQTpN/fddx9jx45lxowZbWWrV69mwoQJFBYWUlhYyNatW9teS2Q5gkceeUTLEYgMECUE6Tf33nsvNTU1MeWPPvoodXV11NXVceuttwKJL0fQ0NCg5QhEBogSQi8lozWc6Ytzfe1rX+OKK65IaNtElyNYtmyZliMQGSBKCL2UjNZwti7O9eyzzzJr1izuu+++thUsE12OIDc3t8fLEUD6LEmgYTTJJEoIvZSM1nA2Ls5VUVHB/v37qaurIxQK8dhjjwHJXY4g2DYtliTQMJpkEiWEftaX1nA2Ls41btw4cnJyuOyyy3jggQeora0FEl+OoLGxMaOXI9AwmmQSJYR+lIrWcLq0hDvT3Nzc9vyVV15pGzpJdDmCDRs2ZOVyBIN5GE3SlxJCP+prazjTF+e6++67mTdvHp988gm5ublUV1fzve99j5kzZzJr1izefvttnnnmGaD9cgRlZWdmTBIAABENSURBVGUxyxHcf//95OfnM2XKlHbLERw/fhwuLUdQmZIP2keDfRhN0peWruhHkZUaIbY1fM8997By5UoOHz7c1hrOyclpaw3PmTOHs3t2MGr2grY6mdYafumll2LKli9f3un2iSxHAPBasCRBtixHMG7cuLbnDzzwAAsWhP/PB8swmqQv9RB6KRmt4aGjv5p1rWGJpWE0SVfqIfRSMlrDWpwr+9x999288847fP755+Tm5vLDH/6Qd955h7q6OsyMvLw8nnvuOSDxVT3nz59P/R9rVU/pf0oIIkmkYTTJJBoyEhERQAlBREQCSggiIgIkkBDM7AUzO2pme6LKVptZk5nVBY9bo177vpntM7NPzOyWqPLZZvZR8NpPI9+mNLNhZvbLoFxrsYiIpEgiPYRfAGVxyp9x98LgsRXAzKYRvsphelDnZ2aWE2y/FlgBFASPyHsuB066ez5ai0VEJGW6TQju/msS/6LLImCju7e4+wFgH1BsZiFglLu/F1wjvQG4LapO5EJqrcUikuUGagXY/Px8gGs06pC4vswhPGxmHwZDSqODsgnAoahtGoOyCcHzjuXt6mgtFpHsN1ArwO7btw/gCBp1SFhvE8JaYApQCDQDTwfl8Vr23kV5V3ViC7UWi0jGG+AVYE+iUYeE9SohuPsRd2919y+AnwORL8Q0AhOjNs0FDgfluXHK29XRWiwig1cyVoANaNQhQb1KCMGcQMTthG/OAfAqsCS4cmgy4cnjWndvBs6Y2dwgUy8DtkTViaRzrcUiMghpBdj00O3SFWb2EnAj8BUzawR+ANxoZoWEg3wQ+DaAu+81s5eBeuAi8JC7twZvVUH4iqUvA28ED4Bq4EUz20e4Z6C1WEQGmWSsABvVg9CoQ4K6TQjufnec4uoutn8CeCJO+S7CC3B1LD8P3NndfohI9urr0vHxVoCdN28ewGg06pAwLW4nIgNqoFaADS47/SpaOj5hSggiMqAGagVYADP72N0/6+s+DxZay0hERAAlBBERCSghiIgIoIQgIiIBJQQREQGUEEREJKCEICIigBKC9KN469yfOHGC0tJSCgoKKC0tbVu0DHq/zj0wQ3fXE+l/Sgi9lIyT34m3nsvok1+8de6rqqooKSmhoaGBkpISqqqqgL6tc094MUXdXU+knykh9FIyTn4XThzO6JNfvHXuo9emLy8vZ/PmzW3lfVznPiPurqdek2QSJYReSsbJb8SMr2f0yS+eI0eOtC1aFgqFOHr0KND3de4z5e566jVJJlFC6Ed9PfnljBzT45Nfupz4emqwrHOvXpNkEiWEAZDMk1+6nPg6M27cOJqbm4HwEsdjx44F+rbOPWT23fUGe69J0pcSQj/q68mv9czxrDv5RdamB1i/fj2LFi1qK9+4cSMtLS0cOHCgbZ37UCjUts69u8dd5z6QdXfXGyy9JklfSgj9qK8nv7N7dmT0ye/uu+9m3rx5fPLJJ+Tm5lJdXU1lZSXbtm2joKCAbdu2UVkZXpo+ep37srKymHXu77//fvLz85kyZQrDr7q0zv3x48chfKOllWToOvfqNUm60v0QeineTT4qKytZvHgx1dXVTJo0qW1N9kRv8jF09NR2J7+lS5fCpZNf2t9aNN469wDbt2+PW97bde7NbI+7F/fDLqdEJNlXVlbGNBx6e3ewQEY0HCR9KSH0UjJOfnnBiQ+y5+Q32CWj4RDv7mBkUMNB0le3CcHMXgAWAEfdfUZQdgXwSyAPOAgsdveTwWvfB5YDrcAj7v5mUD4b+AXwZWAr8B13dzMbBmwAZgPHgbvc/WC/fUKRFFKvSTJJInMIvwDKOpRVAtvdvQDYHvyMmU0j3EKZHtT5mZnlBHXWAiuAguARec/lwEl3z0fXUYtkvYH6sl5wT+Vr9GW9xHWbENz918ROUi0CIgOX64Hboso3unuLux8A9gHFZhYCRrn7e8H45oYOdSLvpeuoRbLcQH1Zb9++fQBHUCMzYb29ymicuzcDBP+ODconAIeitmsMyiYEzzuWt6uj66hFst8Af1nvJGpkJqy/LzuNF3TvoryrOrGFuo5aJCsl68t6gU4bmdJebxPCkWAYiODfo0F5IxD9P5ELHA7Kc+OUt6uj66hFJFqyv6ynUYf2epsQXgUifbJyYEtU+RIzG2ZmkwlPHtcGw0pnzGxu0HVb1qFO5L10HbXIIJSsL+sFOm1katShvW4Tgpm9BLwHXG1mjWa2HKgCSs2sASgNfsbd9wIvA/VADfCQu7cGb1UBPE94onk/8EZQXg2MMbN9ZPC3T0Wk95K4xMlo1MhMWLffQ3D3uzt5qaST7Z8AnohTvovwl2c6lp8H7uxuP0QkOwzUl/WCy06/ihqZCdM3lUVkQA3Ul/UAzOxjd/+sr/s8WGhxOxERAZQQREQkoIQgIiKAEoKIiASUEEREBFBCEBGRgBKCiIgASggiIhJQQhAREUAJQUREAkoIIiICKCGIiEhACUFERAAlBBkgeXl5zJw5k8LCQoqKwssUnzhxgtLSUgoKCigtLeXkyZNt2z/55JPk5+dz9dVXcy64eTrA7t27AaaZ2T4z+6nulSvSf5QQkqC3J7+mn387q09+b7/9NnV1dezatQuAqqoqSkpKaGhooKSkhKqqKgDq6+vZuHEje/fupaamhhPb1tLaGr7PUkVFBcC/EL4bXwFQlorP0h+UJCXdKCEkSW9OfmPv/GHWnvzi2bJlC+Xl4bunlpeXs3nz5rbyJUuWMGzYMCZPnsyQy0PU1tbS3NzM6dOnAX4X3AFrA3Bbqva/P/QlSfoXg+M4kYGjhDBAEjn5Db38q1l78jMz/uIv/oLZs2ezbt06AI4cOUIoFAIgFApx9OhRAJqampg4cWJb3ZyRX6GpqYmmpiZyc3Oj37YRmDBAH2FA9CRJ/qH506w7TiS1lBCSYCBPfma2wsx2mdmuY8eOJe0z9dW7777Lb3/7W9544w3WrFnDr3/96063jXf7WzOLWw7ELcyEuPT1OLl45niPkmQmxERSq0+30DSzg8AZoBW46O5FZnYF8EsgDzgILHb3k8H23weWB9s/4u5vBuWzgV8AXwa2At/J5Jtiv/vuu4wfP56jR49SWlrKNddc0+m2fT35ufs6YB1AUVFR2sZs/PjxAIwdO5bbb7+d2tpaxo0bR3NzM6FQiObmZsaOHQtAbm4uhw4daqvbeuZzxo8fT25uLo2NjdFvmwscjvf7MiEufT1O6GGSzISYSGr1Rw/hJncvdPei4OdKYLu7FwDbg58xs2nAEmA64THOn5lZTlBnLbCCLBkD7erkB/T7yS/d/e53v+PMmTNtz3/1q18xY8YMFi5cyPr16wFYv349ixYtAmDhwoVs3LiRlpYWDhw4wMWThykuLiYUCjFy5EiAPw4mTpcBW1LyofpBX4+TISOuyKrjBPp3on3mzJkAMzTRnrhkDBktAtYHz9dzaTxzEbDR3Vvc/QCwDyg2sxAwyt3fy4Yx0L6c/C6c+resPPkdOXKEG264gWuvvZbi4mK+8Y1vUFZWRmVlJdu2baOgoIBt27ZRWVkJwPTp01m8eDHTpk2jrKyMK0oryMkJtx3Wrl0L4d7nPmA/8EZqPlXf9EeS/FLoP/TbcZJX+Tp5wU3qU62/JtqDYbg9ZEEjc6D0aciIcNf0V2bmwHNBl3ScuzcDuHuzmY0Ntp0A7IyqGxnrvBA871iekY4cOcLtt98OwMWLF7nnnnsoKyvj+uuvZ/HixVRXVzNp0iQ2bdoEtD/5HT3VEnPyu/766/MIn/zeIENPfldddRUffPBBTPmYMWPYvn173DqrVq1i1apVAO1OVEGrcW9UjzQj9eU4GTJkCFeUVmCXZddx0pktW7bwzjvvAOGJ9htvvJGnnnqq24n2efPmRd4i0sjMqrgkQ18Twp+7++HgpL/NzP65i23jddm8i/LYNzBbQXhoiUmTJvV0XwdEX05+HVto2XLySzeROB+s+kbK9kFJMr7IRLuZ8e1vf5sVK1Z0OdE+d+7ctrq9nWgnzc8pA6lPCcHdDwf/HjWzV4Bi4IiZhYLeQQg4GmzeCEyMqh4Z62wMnncsj/f7NCk2iKTLEEY6yfaYaKI9tXo9h2Bmf2xmIyPPgb8gPF73KlAebFbOpfHMV4ElZjbMzCYTHterDYaXzpjZ3EwfKxeRvunNRHtk/iNbJ9oHUl8mlccB/2hmHwC1wOvuXgNUAaVm1gCUBj/j7nuBl4F6oAZ4yN1bg/eqAJ4nwycK01G2tygle/R2ot0vXmi7ICN6on3nzrYpy0HZyOzNhQK9HjJy98+Aa+OUHwdKOqnzBPBEnPJdwIze7ouIZL7eTrSv/p8VcFlOzET7vffeC+HzyvOokZmQvk4qi4j0i95OtP/8TGFMeVFREXv27MHM9rj7w/2+s1lKS1eIiAighCAiIgElBBERAZQQRCRKOi1hIQNPCUEkA+hEnRqDLeZKCINApp1MBmp/MykmEp/+D/uXEkKa0IEtIqmWNd9DSIcFy0REUq0vjcusSQiS+dRLSh9qYA1OSgiDiP7IY2VCTJQoYykmyaGEICLShUxoNED/JElNKktaSHWLLx2vxEqHfUr175eBlXU9hEzJ5hGp+IPLtBgNNjoJd06xia+/4pJ1CUESl1f5esqTQrr9gStZxur4f5SK2KTDcTIYjg0lhBRJhwMcBv4gT5fP3Z3o/RyI2GRKXGDgGxLpFpt0Sgz9HZusTQjp9J+WCZLRCky3P+TeUmxidbb/fYlNx7/ZdI/RQDcaOv7OZMjahBCRDsMi0dL9II/o6X5myh9xf+jqM0Yfa4MhFh31x2fOxLglc1htIOOR9QkBUj8GmokHeE8Nhs+YCMVBIHOPg0GREDqK95/VsYXbXdLobLtMPRBERNImIZhZGfATIAd43t2rBvL3dzyRJ3piH4AEMMrMPiFFcUlTikksxaSDmpoagBlmtg/FJCFp8cU0M8sB1gDzgWnA3WY2LbV7lXqtra0Ak1Bc2igmsRSTWK2trTz00EMAn6KYJCwtEgJQDOxz98/c/Q/ARmBRivcp5WprawFaFJdLFJNYikms2tpa8vPzAf6gmCQuXYaMJgCHon5uBOZ03MjMVgArgh/PBl1kgK8Anyd1DweQPdX2eUYD46NeiomLYtKjYyWrYgJtcWml9zGBLItLVExGAX8aFCsmlz7Pn3a2XbokBItT5jEF7uuAdTGVzXa5e1EydiwVIp/HzO4Ebunwcru4KCZAgsdKtsUEwp8JeIpexiTyHtkUl+iYuPv9US8N6pgk8nnSZcioEZgY9XMucDhF+5JOFJdYikksxSSWYtIL6ZIQ/gkoMLPJZvYlYAnwaor3KR0oLrEUk1iKSSzFpBfSYsjI3S+a2cPAm4Qvm3vB3ff24C1iunwZbh30OS6KSSfvkWXW6e8nhmISK6HPY+4xw2oiIjIIpcuQkYiIpJgSgoiIABmeEMyszMw+MbN9ZlaZ6v3pKzN7wcyOmtmePryHYhL/fRSX2PdQTGLfY3DHxN0z8kF4omg/cBXwJeADYFqq96uPn+lrwHXAHsWkf2KiuCgmikniMcnkHkLWLXfh7r8GTvThLRST+BSXWIpJrEEfk0xOCPGWu5iQon1JF4pJfIpLLMUk1qCPSSYnhISWuxhkFJP4FJdYikmsQR+TTE4I+mp6LMUkPsUllmISa9DHJJMTgr6aHksxiU9xiaWYxBr0McnYhODuF4HIV9M/Bl72nn01Pe2Y2UvAe8DVZtZoZst7Ul8xiU9xiaWYxFJMtHSFiIgEMraHICIi/UsJQUREACUEEREJKCGIiAighCAiIgElBBERAZQQREQk8P8Da6saIrBwVukAAAAASUVORK5CYII=\n"
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "def sigmoid(x):\n",
    "    return 1/(1+np.exp(-x))\n",
    "x = np.random.randn(1000,100) # 1000个数据\n",
    "node_num = 100 # 各隐藏层的节点数\n",
    "hidden_layer_size = 5 # 隐藏层数\n",
    "activations = {} # 激活值的结果保存在这里\n",
    "\n",
    "for i in range(hidden_layer_size):\n",
    "    if i!=0:\n",
    "        x = activations[i-1]\n",
    "    w = np.random.randn(node_num,node_num)*1\n",
    "\n",
    "    z = np.dot(x,w)\n",
    "    a = sigmoid(z) # sigmoid函数\n",
    "    activations[i] = a\n",
    "\n",
    "# 绘制直方图\n",
    "for i ,a in activations.items():\n",
    "    plt.subplot(1,len(activations),i+1)\n",
    "    plt.title(str(i+1)+'-layer')\n",
    "    plt.hist(a.flatten(),30,range=(0,1))\n",
    "plt.show()"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-10-02T07:20:20.544306600Z",
     "start_time": "2023-10-02T07:20:19.351458600Z"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "outputs": [
    {
     "data": {
      "text/plain": "<Figure size 432x288 with 5 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEICAYAAABfz4NwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3df3RV5Z3v8fe3YNFVscUfwcCB8uMElB+KJUZY09ur0mhoHZCpQmzLjyksZqFc29u56zYO99527lSkXdNp6wzl3rTpGHTaCG1HHH/EpmhXV6k0BcWWH6VhjLckRFDAgg5ECd/7x35OOMk5hENIcs5JPq+1zsrJ9+xnZ58vId+zn/3s5zF3R0RE5H3ZPgAREckNKggiIgKoIIiISKCCICIigAqCiIgEKggiIgKoILQzs9fM7OPZPo5co7ykUk5SKSfp5Vte+nVBMLOVZrbNzFrN7JFsH08uMLMhZlZlZv/PzI6b2ctmNjvbx5VtZvaYmbWY2TEz+4OZLcv2MeUKMysys5Nm9li2jyUXmNnPQz7eDo+92T6mntKvCwJwAPgq8P1sH0g6ZjY4Cz92MLAf+M/AB4H/CWwwszFZOJa0spSXh4Ax7n4ZMAf4qplNz8JxpJWlnCSsBX6TxZ+fVpZzstLdLw2PiVk8jhQXkpd+XRDc/Sfu/gRw+HzamVmJmb1oZm+FT43/ZGbvD6+tNbNvdNr+38zsC+H5CDP7sZm9YWaNZnZ/0nZfMbMfhU+jx4AlF/wmz5O7v+PuX3H319z9tLs/BTQC5/zj18/zssvdWxPfhsf4c7XrzzkJx1EOvAVsPo82/Ton3ZUXeXH3fv8gOkt45BzbvAZ8PDyfDswg+jQ9BtgDfCG8VkJ05vG+8P2VwH8Aw4kK7HbgfwHvB8YBrwK3h22/ArwH3Bm2vSQHcjMcOAlcM9DzAnwnHLMDLwGXDuScAJcBfwBGheN5TP9/HODnwBvAm8AW4Ob+kpd+fYbQXe6+3d23uvspd38N+L9EXSy4ez3wJ2BW2Lwc+Lm7HwRuBK5y9//t7u+6+6vAd8M2CS+6+xMefTo/0VfvKR0zuwj4F6Da3X9/ru37e17c/V5gKPCfgJ8ArV236Pc5+Tugyt33n0+jfp4TgC8R/VEeCVQC/2Zm5zybzIe8DMiCYGbPJl0Q+kya1yeY2VNm9no4BVtNVLETqoHPhuefBR4Nzz8MjAinhG+Z2VvA3xBV+YTz+s/VW8zsfUTH/S6wMsQGfF7cvc3dfwnEgBUDNSdmNg34OPDNNK8NyJwkuPuv3f24u7e6ezXRWcIn+kNesnlRJmvc/VyjatYBLwP3uPvx0I93V9LrjwE7zex64FrgiRDfDzS6e1FXP76bh91jzMyAKqJfqE+4+3ugvHQyGBg/gHNyM1G3xh+jXxcuBQaZ2SR3/8g52vbXnJyNA9Yfflf69RmCmQ02s4uBQUS/zBdbZlfghwLHgLfN7BpgRfKL7t5ENOriUeDHSado9cAxM/uSmV1iZoPMbIqZ3dhjb6pnrCP6hfvz8zy97Jd5MbMCMys3s0vDsd0O3AM8n0HzfpkToq6Q8cC08Pg/wNPA7Rm07a85wcw+ZGa3J/6WhDOBjwHPZdA85/PSrwsC8D+AE0AF0SnYiRA7l/8GfBo4TtRX93iabaqBqZw5rcPd24A/J/oP1Eh00el7RMM7c4KZfRj4K6JjfL2rU9w0+mtenOg/ZxNwFPh7oot9mzJo2y9z4u7/4e6vJx7A28BJd38jg+b9MifBRUSDVBIXlf8LcKe7Z3IvQs7nxcJVajlPZvYxolO8Me5+OtvHkyuUl1TKSSrlJL1s56W/nyH0ijA65/PA9/TLfIbykko5SaWcpJcLeVFBOE9mdi3RjTqFwLeyfDg5Q3lJpZykUk7Sy5W8qMtIREQAnSF0Wxht8CMz+72Z7TGzmWZ2uZnVmVlD+DosafsHzGyfme0No1gS8elm9rvw2sNhSGhiErrHQ/zXlkNzDYlI/5S3ZwhXXnmljxkzJms/v7GxkaFDh3LllVdy+vRpTp8+zeuvv87gwYO5+uqref311zl16hSxWIwTJ07Q2NjINddcw3vvvccf/vAHpkyZgpmxZ88eRo0axQc+8AH27dtHQUEBH/zgBzl06BAnTpzgzTffTIxkmOfuC7o6pmznpK9s3779TXe/KtPtB0JelJP0zicvygn5O5fR9OnTPVv+9Kc/+ZgxY/z06dMd4hMmTPADBw64u/uBAwd8woQJ7u6+evVqX716dft2t912m//qV7/yAwcO+MSJE9vjP/jBD3z58uUdtgG2Ed0k9SahgJ/tkc2c9CVgm+fJ70pfUU7SO5+8KCeay6hbXn31Va666ir+8i//khtuuIFly5bxzjvvcPDgQQoLCwEoLCzk0KFDADQ3NzNq1Kj29rFYjObmZpqbm4nFYinxzm3c/RTRPCdXdD4WM1tu0ZoP2954I5Mh4iIi6akgdMOpU6d46aWXWLFiBS+//DIf+MAHWLNmzVm39zTdcmZ21vjZ2pDm9nR3r3T3YncvvuqqjHsMRERSqCB0QywWIxaLcdNNNwFw11138dJLLzF8+HBaWloAaGlpoaCgoH37/fvPzD3V1NTEiBEjiMViNDU1pcQ7twnTbXwQONIHb09EBigVhG64+uqrGTVqFHv3Rnerb968mUmTJjFnzhyqq6sBqK6uZu7cuQDMmTOHmpoaWltbaWxspKGhgZKSEgoLCxk6dChbt27F3Vm/fn2HNol9EU2A9byf5bRBRKQnDMjZTnvCP/7jP/KZz3yGd999l3HjxvHP//zPnD59mvnz51NVVcXo0aPZuHEjAJMnT2b+/PlMmjSJwYMHs3btWgYNGgTAunXrWLJkCSdOnGD27NnMnh1NmLh06VIWLlwIMAX4Ih3nPhcR6XEqCN00bdo0tm3blhLfvDn9SoOrVq1i1apVKfHi4mJ27tyZEr/44ovZuHEjZrbT3Usu/IhFRLqmLiPpUWPGjGHq1KlMmzaN4uJiAI4cOUJpaSlFRUWUlpZy9OjR9u0feugh4vE4EydO5LnnzswgvH37dqZOnUo8Huf+++9vv8je2toKME437In0PBUE6XEvvPACO3bsaD+DWrNmDbNmzaKhoYFZs2a1j8javXs3NTU17Nq1i9raWu69917a2toAWLFiBZWVlTQ0NNDQ0EBtbS0AVVVVAKfcPU60mtfX+v4divRPKgjS6zZt2sTixYsBWLx4MU888UR7vLy8nCFDhjB27Fji8Tj19fW0tLRw7NgxZs6ciZmxaNGiDm2Aw2HXPwJmJab7EJELo4JwHsZUPM2YiqezfRg5pXNOzIzbbruN6dOnU1lZCdDjN+wRrQOdkzfs6ffj7JSbVLn2NyXjghCWbXvZzJ4K3w+Yidxy7R8tl23ZsoWXXnqJZ599lrVr1/KLX/zirNsOlBv2dF0lVV/kZMGCBQBT8iUnueB8zhA+D+xJ+r4C2OzRws+bw/eY2SSiIZKTgTLgO2Y2KLRZBywHisKjLMSXAkfVL5z/EjfWFRQUMG/ePOrr63v8hj3g/ZBfN+zpukqq3s7JsGHDAHaSRznJtowKgpnFgE8SreOZMJdo/U/C1zuT4jXu3urujcA+oMTMCoHL3P3FcIPV+k5tEvtSv3Ceeueddzh+/Hj785/+9KdMmTKlx2/Y40wXUd7esKfrKql6OieJfZHHOelrmZ4hfAv470Dysm7D3b0FIHwtCPGRwP6k7ZpCbGR43jneoU1X/cKS2w4ePMhHP/pRrr/+ekpKSvjkJz9JWVkZFRUV1NXVUVRURF1dHRUVFUDHG/bKyspSbthbtmwZ8Xic8ePHd7hhDxhsZvuIbtiryMqb7YKuq5xbX+REk0Oev3PemGZmdwCH3H27md2cwT7TVWHvIt5Vm87Hspyoy4nRo0dncCjSl8aNG8crr7ySEr/iiit69IY94FV3L77Q4+0rW7ZsYcSIERw6dIjS0lKuueaas27b29dVgEqA4uLirJ5VKSe5KZMzhD8D5pjZa0ANcKuZPQYcDN1AhK+HwvZNwKik9jHgQIjH0sQ7tOmqXziXLhSKZErXVVL1RU40OeT5O2dBcPcH3D3m7mOILhY/7+6fBZ4EEp10i4FN4fmTQHkYOTSW6OJxfehWOm5mM0Jf3qJObRL7ytt+YZHOdF0lVV/lRJNDnr8LmctoDbDBzJYCfwTuBnD3XWa2AdgNnALuc/e20GYF8AhwCfBseABUAY+GfuEjaCI36ScOHjzIvHnzgGgdjU9/+tOUlZVx44039uhEiCtXrhycq/9/Og/Z7qucaHLI83deBcHdfw78PDw/DMw6y3YPAg+miW8j+gfqHD9JKCgi/Ymuq6Tqq5xocsjzpzuVRUQEUEEQEZFA6yGIiPSxXJ0KR2cIIiICqCCIiEiggiAiIoAKgoiIBCoIIiICqCCIiEiggiAiIoAKgoiIBCoIIiICqCCIiEiggiAiIoAKQreNGTOGqVOnMm3aNIqLo1mHjxw5QmlpKUVFRZSWlnL06NH27R966CHi8TgTJ07kueeea49v376dqVOnEo/Huf/++9uX/mttbWXBggUAU8zs12Y2pu/enYgMRCoIF+CFF15gx44dbNu2DYA1a9Ywa9YsGhoamDVrFmvWrAFg9+7d1NTUsGvXLmpra7n33ntpa4vWDFqxYgWVlZU0NDTQ0NBAbW0tAFVVVQwbNgxgJ/BN4Gt9/w5FZCBRQehBmzZtYvHiaCXQxYsX88QTT7THy8vLGTJkCGPHjiUej1NfX09LSwvHjh1j5syZmBmLFi3q0CaxL+BHwCxLrCAuItILzlkQzOxiM6s3s1fMbJeZ/W2If8XMms1sR3h8IqnNA2a2z8z2mtntSfHpZva78NrDiT9wYf3lx0M8L7pHzIzbbruN6dOnU1lZCURLAxYWFgJQWFjIoUOHAGhubmbUqFHtbWOxGM3NzTQ3NycWSO8Q79zG3U8Bf+LMurnJx7HczLaZ2bY33nijV96riAwMmayH0Arc6u5vm9lFwC/NLLEW8jfd/e+TNzazSUTrl04GRgA/M7MJYV3ldcByYCvwDFBGtK7yUuCou8fNrJyoe2TBhb+93rNlyxZGjBjBoUOHKC0t5ZprrjnrtunW9jazs8bP1gZICbp7JVAJUFxcrEXERaTbznmG4JG3w7cXhUdXf3jmAjXu3urujcA+oMTMCoHL3P1Fj/7arQfuTGpTHZ7nRffIiBEjACgoKGDevHnU19czfPhwWlpaAGhpaaGgoACIPvnv37+/vW1TUxMjRowgFovR1NSUEu/cxswGAx8kWkBdRKRXZHQNwcwGmdkO4BBQ5+6/Di+tNLPfmtn3zWxYiI0E9ic1bwqxkeF553iHNvnQPfLOO+9w/Pjx9uc//elPmTJlCnPmzKG6Oqpr1dXVzJ07F4A5c+ZQU1NDa2srjY2NNDQ0UFJSQmFhIUOHDmXr1q24O+vXr+/QJrEv4C7geT/LaYOISE/IaAnN0N0zzcw+BPyrmU0h6v75O6Kzhb8DvgF8Dkj3yd67iHOO15KPIye6Rw4ePMi8efMAOHXqFJ/+9KcpKyvjxhtvZP78+VRVVTF69Gg2btwIwOTJk5k/fz6TJk1i8ODBrF27lkGDBgGwbt06lixZwokTJ5g9ezazZ88GYOnSpSxcuBBgCvBFom44EZFec15rKrv7W2b2c6As+dqBmX0XeCp82wSMSmoWAw6EeCxNPLlNUz50j4wbN45XXnklJX7FFVewefPmtG1WrVrFqlWrUuLFxcXs3LkzJX7xxRezceNGzGynu5dc+FGLiHQtk1FGV4UzA8zsEuDjwO/DNYGEeUTj5QGeBMrDyKGxQBFQ7+4twHEzmxGuDywCNiW1SYyxVPeIiEgWZHKGUAhUm9kgogKywd2fMrNHzWwaUdfOa8BfAbj7LjPbAOwGTgH3hS4ngBXAI8AlRKOLEqOVqoBHzWwf0ZmBukdERPrYOQuCu/8WuCFNfGEXbR4EHkwT30bUJ945fhK4+1zHIiIivUd3KouICKCCICIigQqC9Ki2tjZuuOEG7rjjDkAzwIrkExUE6VHf/va3ufbaa9u/1wywIvlDBUF6TFNTE08//TTLli1rj2kGWJH8oYIgPeYLX/gCX//613nf+878WmVjBljInWlOQN1oZ6O85B4VBOkRTz31FAUFBUyfPj2j7XtzBtiwbaW7F7t78VVXXZXRMfUWdaOlp7zkHhUE6RFbtmzhySefZMyYMZSXl/P888/z2c9+dsDPAKtutPSUl9ykgiA94qGHHqKpqYnXXnuNmpoabr31Vh577LEBPwOsutHSy5W85FJOcoEKgvSqiooK6urqKCoqoq6ujoqKCqDjDLBlZWUpM8AuW7aMeDzO+PHjO8wAe/jwYTgzA2xFVt5UhtSNll4u5SVXcpIrzmu2U5FM3Hzzzdx8883AwJ4BNtGN9swzz3Dy5EmOHTvWoRutsLBwQHajKS+5S2cIIr1E3WjpKS+5SwVBpI8N5G60rigv2acuI5E+oG609JSX3KIzBBERAVQQREQkyGQJzYvNrN7MXjGzXWb2tyF+uZnVmVlD+Dosqc0DZrbPzPaa2e1J8elm9rvw2sOJG0XCcpuPh7huMxcRyYJMzhBagVvd/XpgGlBmZjOILtJsdvciYHP4HjObRLQE5mSgDPhOWH4TYB2wnGid5aLwOsBS4Ki7x9Ft5iIiWXHOguCRt8O3F4WHA3OBxLiuauDO8HwuUOPure7eCOwDSsysELjM3V8Mw7/Wd2qT2JduMxcRyYKMriGY2SAz2wEcAurc/dfAcHdvAQhfC8LmI4H9Sc2bQmxkeN453qGNbjMXEcmOjAqCu7e5+zQgRvRpf0oXm6f7ZO9dxLtq0/k4dJu5iEgvOa/7ENz9LTP7OVHf/0EzK3T3ltAddChs1gSMSmoWAw6EeCxNPLlNk24zzw9jKp7O9iGISA/LZJTRVWb2ofD8EuDjwO+BJ4HE/LKLgU3h+ZNAeRg5NJbo4nF96FY6bmYzwvWBRZ3aJPal28xFRLIgkzOEQqA6jBR6H7DB3Z8ysxeBDWa2FPgjcDeAu+8ysw3AbuAUcJ+7t4V9rQAeAS4Bng0PgCrgUTPbR3RmUN4Tb05ERDKXySij37r7De5+nbtPcff/HeKH3X2WuxeFr0eS2jzo7uPdfaK7P5sU3xb2Md7dVybOAtz9pLvf7e5xdy9x91d74832NC0BKCL9ie5UvgBaAlBE+hMVhG7SEoAi0t+oIHSTlgAUkf5GBaEbtASgiPRHWg+hG7QEoIj0RzpD6AYtASgi/ZEKQg/SEoAiks/UZXSBtASgiPQXOkMQERFABUFERAIVBBERAVQQREQkUEEQERFABUFERAIVBBERAVQQREQkyGQJzVFm9oKZ7TGzXWb2+RD/ipk1m9mO8PhEUpsHzGyfme01s9uT4tPN7HfhtYcT0zmH5TYfD3EtBiMikgWZnCGcAv7a3a8FZgD3mdmk8No33X1aeDwDEF4rByYDZcB3wvKbAOuA5UTrLBeF1wGWAkfdPY4WgxERyYpMltBscfeXwvPjwB5gZBdN5gI17t7q7o3APqDEzAqBy9z9xTBJ23rgzqQ2iZnctBhMHjp58iQlJSVcf/31TJ48mS9/+cuAlhUVySfndQ0h/Ae8Afh1CK00s9+a2ffNbFiIjQT2JzVrCrGR4XnneIc2WgwmPw0ZMoTnn3+eV155hR07dlBbW8vWrVu1rKhIHsm4IJjZpcCPgS+4+zGi7p/xwDSgBfhGYtM0zb2LeFdtOga0GEzOMjMuvfRSAN577z3ee+89zEzLiorkkYwKgpldRFQM/sXdfwLg7gfdvc3dTwPfBRIzcjYBo5Kax4ADIR5LE+/QRovB5K+2tjamTZtGQUEBpaWl3HTTTVlZVjSX9FVXGjAuXwZlqHsxd2UyysiAKmCPu/9DUrwwabN5RKfxAE8C5WHk0Fiii8f17t4CHDezGWGfi4BNSW0SH/20GEyeGjRoEDt27KCpqYn6+vq003on9OayoqFNTnQv9lVXGnAqXwZlqHsxd2VyhvBnwELg1k5DTL8ehpD+FrgF+K8A7r4L2ADsBmqB+9y9LexrBfA9ogvN/w48G+JVwBVmtg8tBpP3PvShD3HzzTdTW1vbvqwo0KfLiuZK92JfdaUBh8OPzPmuNHUv5q5MRhn90t3N3a9LHmLq7gvdfWqIzwlnAIk2D7r7eHef6O7PJsW3ufuU8NrKxFmAu59097vdPe7uJe7+au+8Xektb7zxBm+99RYAJ06c4Gc/+xnXXHONlhWlb7rSgHchfwZl5Er3Yi7lJBdoxTTpES0tLSxevJi2tjZOnz7N/PnzueOOO5g5cybz58+nqqqK0aNHs3HjRqDjsqKDBw9OWVZ0yZIlnDhxgtmzZ3dYVnThwoVwZlnR8qy82fOU6Ep76623mDdvXta60ty9EqgEKC4uzmohVU5ykwqC9IjrrruOl19+OSWuZUXPSNeVVlhY2CNdabt3734/5N+gjN7MSabdi3KG5jIS6UV91ZXGme6QnO9KU/di7tIZgkgv6quutJUrVw4OgzKOkONdaepezF0qCCK9qK+60oBX3b34Qo+3L6h7MXepy0hERAAVBBERCVQQREQEUEEQEZFABUFERAAVhG7RbI0i0h+pIHSDZmsUkf5IBaEbNFujiPRHKgjdpNkaRaS/UUHoplxZDCZX5v0XkfyngnCBcmExGBGRnqCC0A2arVFE+qNM1lQeZWYvmNkeM9tlZp8P8cvNrM7MGsLXYUltHggLfu81s9uT4tPDspv7zOzhxEXSsP7y4/mySHhLSwu33HIL1113HTfeeCOlpaXccccdVFRUUFdXR1FREXV1dVRURCuBJs/WWFZWljJb47Jly4jH44wfP77DbI2HDx+GM7M1allREelVmcx2egr4a3d/ycyGAtvNrA5YAmx29zVmVkH0B+tLZjaJaKrZycAI4GdmNiGsq7wOWA5sBZ4ByojWVV4KHHX3uJmVEw2xXNCTb7QnabZGEemPMllTucXdXwrPjwN7gJHAXCDRp1EN3BmezwVq3L3V3RuBfUCJmRUCl7n7i6HrY32nNol9aYiliEgWnNc1hNCVcwPwa2C4u7dAVDSAgrDZSGB/UrOmEBsZnneOd2ijIZYiItmRcUEws0uBHwNfcPdjXW2aJuZdxLtq0zGgIZYiIr0mo4JgZhcRFYN/cfefhPDB0A1E+HooxJuAUUnNY8CBEI+liXdooyGWIiLZkckoIwOqgD3u/g9JLz0JJOZWWAxsSoqXh5FDY4EioD50Kx03sxlhn4s6tUnsS0MsRUSyIJNRRn8GLAR+Z2Y7QuxvgDXABjNbCvwRuBvA3XeZ2QZgN9EIpfvCCCOAFcAjwCVEo4ueDfEq4NF8WSRcRKQ/OmdBcPdfkr6PH2DWWdo8CDyYJr6NaFx95/hJQkEREZHs0J3KIiICqCCIiEiggiAiIoAKgoiIBCoIIiICqCCIiEiggiAiIoAKgoiIBCoIIiICqCBID9m/fz+33HIL1157LZMnT+bb3/42AEeOHKG0tJSioiJKS0s5evRoe5uHHnqIeDzOxIkTee6559rj27dvZ+rUqcTjce6//34S01q1trayYMECgCn5sLKeSL5RQZAeMXjwYL7xjW+wZ88etm7dytq1a9m9ezdr1qxh1qxZNDQ0MGvWLNasWQPA7t27qampYdeuXdTW1nLvvffS1hZNebVixQoqKytpaGigoaGB2tpaAKqqqhg2bBjATuCbRCvr5bS+KpTAuHxZglYfHnKXCoL0iMLCQj7ykY8AMHToUK699lqam5vZtGkTixdHE9kuXryYJ554AoBNmzZRXl7OkCFDGDt2LPF4nPr6elpaWjh27BgzZ87EzFi0aFGHNol9kScr6/VVoQROuXucPCiU+vCQu1QQpMe99tprvPzyy9x0000cPHiQwsJCICoahw5Fy2Y0NzczatSZZTNisRjNzc00NzcTi8VS4p3bdLWyHuTO6np9VSiBw+FH5nyh1IeH3KWCID3q7bff5lOf+hTf+ta3uOyyy866XbrlLszsrPGztSHNynph25xbXa83CyXwLpy7UOaaXPjwIGeoIEiPee+99/jUpz7FZz7zGf7iL/4CgOHDh9PS0gJAS0sLBQXR0tuxWIz9+88svd3U1MSIESOIxWI0NTWlxDu3ybeV9XKhUObKWVOCcpJ7VBCkR7g7S5cu5dprr+WLX/xie3zOnDlUV1cDUF1dzdy5c9vjNTU1tLa20tjYSENDAyUlJRQWFjJ06FC2bt2Ku7N+/foObRL7Io9W1uuLQgm8H7oulLl01pQrHx5yKSe5IJMlNL9vZofMbGdS7Ctm1mxmO8LjE0mvPRBGO+w1s9uT4tPN7HfhtYcT/Xlhqc3H82WEhKS3ZcsWHn30UZ5//nmmTZvGtGnTeOaZZ6ioqKCuro6ioiLq6uqoqKgAYPLkycyfP59JkyZRVlbG2rVrGTRoEADr1q1j2bJlxONxxo8fz+zZswFYunQphw8fhmiRpS8CFVl5s+ehrwolZ7pDcr5Q6sND7spkCc1HgH8C1neKf9Pd/z45YGaTiJa/nAyMAH5mZhPCEprrgOXAVuAZoIxoCc2lwFF3j5tZOdFogAXdfkeSFR/96EfPdprO5s2b08ZXrVrFqlWrUuLFxcXs3LkzJX7xxRezceNGzGynu5dc2BH3jUShnDp1KtOmTQNg9erVVFRUMH/+fKqqqhg9ejQbN24EOhbKwYMHpxTKJUuWcOLECWbPnt2hUK5cuXJwvixB21c5WbhwIZz58JDTOckVmSyh+Yvz+NQ+F6hx91agMfyClpjZa8Bl7v4igJmtB+4kKghzga+E9j8C/snMTNVc+oO+KpTAq+5efCHH2lf04SF3Xcg1hJVm9tvQpTQsxEYC+5O2aQqxkeF553iHNvkylFBEpD/qbkFYB4wHpgEtwDdCPN04X+8i3lWb1GCOXADSnZYi0h91qyC4+0F3b3P308B3gcQpWRMwKmnTGHAgxGNp4h3a5MtQQt1pKSL9UbcKgpkVJn07j+iPFsCTQHkYOTQWKALq3b0FOG5mM8LookXApqQ2iVsK82I0gO60FI8CpU8AAAtYSURBVJH+6JwXlc3sh8DNwJVm1gR8GbjZzKYRde28BvwVgLvvMrMNwG7gFHBfGGEEsIJoxNIlRBeTnw3xKuDRfBkh0Vmmd1rOmDGjvU3ijsqLLroo4zstzSxxbeXN5J9vZsuJRm8xevToXnufItL/ZTLK6J404aoutn8QeDBNfBvRELDO8ZPA3ec6jlyUC3daunslUAlQXFyc02dWIpLbdKdyN+XKnZYiIj1FBaEbdKeliPRHmdypLJ3oTksR6Y9UELpBd1qKSH+kLiMREQFUEEREJFBBEBERQAVBREQCFQQREQFUEEREJFBBEBERQAVBREQCFQQREQFUEEREJFBBEBERQAVBRESCcxYEM/u+mR0ys51JscvNrM7MGsLXYUmvPWBm+8xsr5ndnhSfbma/C689nFgOMiy3+XiIazF5EZEsyeQM4RGgrFOsAtjs7kXA5vA9ZjaJaJrmyaHNd8xsUGizjmipx6LwSOxzKXDU3eNoMXkRkaw5Z0Fw91+QulLXXCCxeks1cGdSvMbdW929EdgHlJhZIXCZu78YFnlZ36lNYl9aTF5EJEu6ew1huLu3AISvBSE+EtiftF1TiI0MzzvHO7Rx91NAYjH5FGa23My2mdm2N954o5uHLiIi6fT0ReV0n+y9i3hXbVKD7pXuXuzuxVdddVU3D1FERNLpbkE4GLqBCF8PhXgTMCppuxhwIMRjaeId2mgxeRGR7OluQXgSWByeLwY2JcXLw8ihsUQXj+tDt9JxM5sRrg8s6tQmsS8tJi8ikiXnXFPZzH4I3AxcaWZNwJeBNcAGM1sK/BG4G8Ddd5nZBmA3cAq4z93bwq5WEI1YugR4NjwAqoBHzWwf0ZmBFpMXEcmCcxYEd7/nLC/NOsv2DwIPpolvA6akiZ8kFBQREcke3ancDWMqns72IeSkz33ucxQUFDBlypm6f+TIEUpLSykqKqK0tJSjR4+2v/bQQw8Rj8eZOHEizz33XHt8+/btTJ06lXg8zv3330+iB7G1tZUFCxYATNFNjCI9TwWhC2MqntYf//OwZMkSamtrO8TWrFnDrFmzaGhoYNasWaxZswaA3bt3U1NTw65du6itreXee++lrS3qXVyxYgWVlZU0NDTQ0NDQvs+qqiqGDRsGsJM8uYlRRTJVX+UkHo8DXJMPOckVKgjSYz72sY9x+eWXd4ht2rSJxYujMQOLFy/miSeeaI+Xl5czZMgQxo4dSzwep76+npaWFo4dO8bMmTMxMxYtWtShTWJf5MlNjCqSqfoqJ/v27QM4SB7kJFeoIEivOnjwIIWFhQAUFhZy6FA0Qrm5uZlRo86MUI7FYjQ3N9Pc3EwsFkuJd26TLzcxqkim6uOcHCUPcpIrVBAkK9KNLDazs8bP1oY8vIlxoBfJdHorJ0Fe5iQbVBCkVw0fPpyWlhYAWlpaKCiIZjmJxWLs339mlpOmpiZGjBhBLBajqakpJd65TX+8iXEgF8mzUU76lgpCN+liYWbmzJlDdXU0d2F1dTVz585tj9fU1NDa2kpjYyMNDQ2UlJRQWFjI0KFD2bp1K+7O+vXrO7RJ7Is8volRRTJVb+UkyMucZIMKQjfpYmGqe+65h5kzZ7J3715isRhVVVVUVFRQV1dHUVERdXV1VFRUADB58mTmz5/PpEmTKCsrY+3atQwaFM2Uvm7dOpYtW0Y8Hmf8+PHMnj0bgKVLl3L48GGI7mf5ImHa9XyjIpmqF3MyjDzNSVa4e14+pk+f7r3tw1966qwPd/fGxkafPHly+/YTJkzwAwcOuLv7gQMHfMKECe7uvnr1al+9enX7drfddpv/6le/8gMHDvjEiRPb4z/4wQ98+fLlHbYBthHdQPgmYJ7lnCSky0dfAbZ5jv2uJHTOSXl5uV999dU+ePBgHzlypH/ve9/zN99802+99VaPx+N+6623+uHDh9vbf/WrX/Vx48b5hAkT/JlnnmmP/+Y3v/HJkyf7uHHj/L777vPTp0+7u/uJEyf8rrvucuAkUA+MS5eDzo+BkJPx48c78E6u5ySX/v+c805lyVxXF8ZmzJjRvl3iAthFF12U8cVCM0tcGHsz+Wea2XKihYcYPXp0r7036Z4f/vCHaeObN29OG1+1ahWrVq1KiRcXF7Nz586U+MUXX8zGjRsxs53uXnJhR9s3+ionAGa2x91fvYDDHVDUZdQHvBcvjLkuiolID1FB6EG6WCgi+UwFoQfpYqGI5DMVhG7SiBoR6W90UbmbdLFQRPobnSGIiAhwgQXBzF4zs9+Z2Q4z2xZil5tZnZk1hK/DkrZ/wMz2mdleM7s9KT497GefmT2siahERPpeT5wh3OLu09y9OHxfAWx29yJgc/geM5tEtDzmZKAM+I6ZDQpt1hGNpS8Kj7IeOC4RETkPvdFlNBdIDI+pBu5Mite4e6u7NwL7gBIzKwQuc/cXwyia9UltRESkj1xoQXDgp2a2PdwxCzDc3VsAwteCEB8JJM841RRiI8PzznEREelDFzrK6M/c/YCZFQB1Zvb7LrZNd13Au4in7kDTNIiI9JoLOkNw9wPh6yHgX4ES4GDoBiJ8PRQ2bwKSV62IAQdCPJYmnu7naZoGEZFe0u2CYGYfMLOhiefAbURTNT8JJNavWwxsCs+fBMrNbIiZjSW6eFwfupWOm9mMMLpoUVIbERHpIxfSZTQc+NcwQnQw8AN3rzWz3wAbzGwp8EfgbgB332VmG4DdwCngPndvC/taATwCXAI8Gx4iItKHul0QwpSy16eJHwZmnaXNg8CDaeLbiKZoEBGRLNGdyiIiAqggiIhIoIIgIiKACoKIiAQqCCIiAqggiIhIoIIgIiKACoKIiAQqCCIiAqggiIhIoIIgIiKACoKIiAQqCCIiAqggiIhIoIIgIiKACoKIiAQqCHLexlQ8ne1DyAtjKp5WriQjufK7kjMFwczKzGyvme0zs4psH08OuUx5SaGcpFJOOqmtrQWYopxkLicKgpkNAtYCs4FJwD1mNimbx3Suat0XFb2trQ1gNDmUl2xTTlIpJ6na2tq47777AP6AcpKxnCgIQAmwz91fdfd3gRpgbpaPKevq6+sBWpWXM5STVMpJqvr6euLxOMC7yknmBmf7AIKRwP6k75uAmzpvZGbLgeXh27fNbG94fiXwZq8e4VnY13plt4n3MwwYkRRPyUuu5KSX8pAs45zAWfOSld+TXs7NlUAb3c9JYh99kpc++D2BMzm5DPhwiOVsTjocT+/lJ/n9fPhsG+VKQbA0MU8JuFcClSmNzba5e3FvHFg2JN6Pmd0N3N7p5Q55UU6ADH9X+ltOIHpPwNfoZk4S++hPeUnOibsvS3ppQOckk/eTK11GTcCopO9jwIEsHUsuUV5SKSeplJNUykk35EpB+A1QZGZjzez9QDnwZJaPKRcoL6mUk1TKSSrlpBtyosvI3U+Z2UrgOWAQ8H1333Ueu0g55ctzlXDBeVFOzrKPfqZS/39SKCepMno/5p7SrSYiIgNQrnQZiYhIlqkgiIgIkOcFob9Nd2Fm3zezQ2a28wL2oZyk34/ykroP5SR1HwM7J+6elw+iC0X/DowD3g+8AkzK9nFd4Hv6GPARYKdy0jM5UV6UE+Uk85zk8xlCv5vuwt1/ARy5gF0oJ+kpL6mUk1QDPif5XBDSTXcxMkvHkiuUk/SUl1TKSaoBn5N8LggZTXcxwCgn6SkvqZSTVAM+J/lcEHRreirlJD3lJZVykmrA5ySfC4JuTU+lnKSnvKRSTlIN+JzkbUFw91NA4tb0PcAGP79b03OOmf0QeBGYaGZNZrb0fNorJ+kpL6mUk1TKiaauEBGRIG/PEEREpGepIIiICKCCICIigQqCiIgAKggiIhKoIIiICKCCICIiwf8H7eHEVzU0T1AAAAAASUVORK5CYII=\n"
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.random.randn(1000,100) # 1000个数据\n",
    "node_num = 100 # 各隐藏层的节点数\n",
    "hidden_layer_size = 5 # 隐藏层数\n",
    "activations = {} # 激活值的结果保存在这里\n",
    "\n",
    "for i in range(hidden_layer_size):\n",
    "    if i!=0:\n",
    "        x = activations[i-1]\n",
    "    w = np.random.randn(node_num,node_num)*0.01\n",
    "\n",
    "    z = np.dot(x,w)\n",
    "    a = sigmoid(z) # sigmoid函数\n",
    "    activations[i] = a\n",
    "\n",
    "# 绘制直方图\n",
    "for i ,a in activations.items():\n",
    "    plt.subplot(1,len(activations),i+1)\n",
    "    plt.title(str(i+1)+'-layer')\n",
    "    plt.hist(a.flatten(),30,range=(0,1))\n",
    "plt.show()"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-10-02T07:25:12.687497500Z",
     "start_time": "2023-10-02T07:25:11.742500200Z"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "outputs": [
    {
     "data": {
      "text/plain": "<Figure size 432x288 with 5 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEICAYAAABYoZ8gAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de3TV5Z3v8ffXJHJHQAgEQho1yN1rlHA6Y2lDEGxHq0ctaC0oliPVqdbTSxzmnDWcM0BwtVOrY+2hQpuqI6OdaU3R0tJYV9dUkaJWDWgIGCq5cIdySQgmPOeP/dthh72T7Ozs7OvntVYWO8/+/XZ++2Hnm+f3XL6POecQEZH0cV68L0BERGJLgV9EJM0o8IuIpBkFfhGRNKPALyKSZhT4RUTSTNoFfjPbbWaz430diUb1Ekx1Ekx1EiwZ6yQlAr+ZPWBmW82sxcx+Gu/rSQRm1s/M1prZX8zsuJm9Y2bz4n1d8WZmz5pZo5kdM7MdZnZvvK8pUZjZBDM7ZWbPxvta4s3MXvPq4oT3VR3va4qmlAj8QAPwz8C6eF9IKGaWGYcfmwnsAT4DXAD8L+AFM8uPw7WEFKd6WQXkO+eGAjcC/2xmV8fhOkKKU534PQn8KY4/P6Q41skDzrnB3tfEOF1DSL2tk5QI/M65/3TO/RI41JPzzOxaM3vDzI56rcB/NbPzveeeNLPvnXP8r8zsIe/xWDP7DzM7YGa1Zvb1gOP+ycx+7rUujwGLev0me8g5d9I590/Oud3OuTPOuQ1ALdBtkEvxetnmnGvxf+t9XdLdealcJ951zAeOApU9OCel6yQSSVMnzrmU+cLX6v9pN8fsBmZ7j68GivC1jvOBD4CHvOeuxXcncZ73/UigCRiN7w/mW8D/Bs4HLgY+Aq73jv0n4BPgi96xAxKgbkYDp4BJ6V4vwA+9a3bA28DgdK4TYCiwAxjvXc+z6f77A7wGHAAOAn8EZqVSnaREiz9Szrm3nHObnXOtzrndwP/D1zWCc24L8Feg2Dt8PvCac24fcA0wyjn3f5xzp51zHwE/9o7xe8M590vna203x+o9hWJmWcBzQLlz7sPujk/1enHOfQ0YAvwt8J9AS9dnpHyd/F9grXNuT09OSvE6+Q6+4DsOWAP8ysy6vTNMljpJ6cBvZr8OGJy5M8Tzl5rZBjPb690+rcT3V9ivHPiy9/jLwDPe408BY73buaNmdhT4B3x/uf169EvUV8zsPHzXfRp4wCtL+3pxzrU55/4LyAWWpmudmNkVwGzg+yGeS8s6AXDOvemcO+6ca3HOleNr9d+QKnUSz4GkPuec624Wy1PAO8AC59xxr6/t1oDnnwWqzOxyYDLwS698D1DrnJvQ1Y+P8LKjxswMWIvvw3ODc+4TUL2cIxO4JI3rZBa+LomPfR8XBgMZZjbFOXdVN+emap2E4gBLlc9JSrT4zSzTzPoDGfg+tP0tvFHvIcAx4ISZTQKWBj7pnKvDN8vhGeA/Am6vtgDHzOw7ZjbAzDLMbJqZXRO1NxUdT+H7cP1dD28NU7JezCzbzOab2WDv2q4HFgCvhnF6StYJvm6MS4ArvK8fAS8D14dxbkrWiZkNM7Pr/XHEa9lfB/wmjNOTok5SIvAD/wg0A6X4bp+avbLufBO4AziOrz/t30McUw5M5+wtGc65NuDv8P2i1OIbAHoa37TJhGBmnwL+B75r3NvV7WkIqVovDt8vYh1wBPguvoG3l8I4NyXrxDnX5Jzb6/8CTgCnnHMHwjg9JesEyMI3UcQ/uPv3wBedc+HM5U+KOjHnEu2OKrGY2XX4bs/ynXNn4n09iUL1Ekx1Ekx1EiwR6iRVWvx9wpsN8yDwtD60Z6legqlOgqlOgiVKnSjwd8LMJuNb0JIDPBbny0kYqpdgqpNgqpNgiVQn6uoREUkzavF3wczWmdl+M6sKKBthZpvMrMb7d3jAc4+Y2U4zq/ZmjPjLrzaz973nHvemWfoTqf27V/6mJVAeHRFJXQnf4h85cqTLz8+Py88+fvw4GRkZ1NbWMnXqVADq6urIzMxkzJgx7N27l9bWVnJzc2lubqa2tpZJkybxySefsGPHDqZNm4aZ8cEHHzB+/HgGDRrEzp07yc7O5oILLmD//v00Nzdz8OBB/8yBm51zX+ruuuJZJ7H01ltvHXTOjQrnWNVJaOlQL6qT0LqsFxfjHBg9/br66qtdPNXW1rqpU6e2f3/ppZe6hoYG55xzDQ0N7tJLL3XOObdy5Uq3cuXK9uPmzJnjXn/9ddfQ0OAmTpzYXv5v//ZvbsmSJR2OAbbiW0h0EO+PcVdf8a6TWAG2uiT5nMRKT+rEpUm9qE5C66pe1NXTQ/v27SMnJweAnJwc9u/fD0B9fT3jx49vPy43N5f6+nrq6+vJzc0NKj/3HOdcK748HheG+rlmtsR8ew5sPXAgnCnWIiKhKfBHiQvRZWZmnZZ3dg6dLMt2zq1xzhU65wpHjQr7rlZEJIgCfw+NHj2axsZGABobG8nOzgZ8Lfk9e87mUKqrq2Ps2LHk5uZSV1cXVH7uOV6KiQuAw7F5JyKSrhT4e+jGG2+kvLwcgPLycm666ab28vXr19PS0kJtbS01NTVce+215OTkMGTIEDZv3oxzjp/97GcdzvG/Fr5ETq+6Tm4DRESiRYG/CwsWLGDmzJlUV1eTm5vL2rVrKS0tZdOmTUyYMIFNmzZRWloKwNSpU7n99tuZMmUKc+fO5cknnyQjIwOAp556invvvZeCggIuueQS5s3zJfhbvHgxhw4dApgGPIwv15CISJ9K6bTMvfX888+HLK+sDL073bJly1i2bFlQeWFhIVVVVUHl/fv358UXX8TMqpxz1/buakVEwqMWv4j0iXvuuYfs7GymTZsW9Nx3v/tdzAzfEhafVatWUVBQwMSJE/nNb85mQH7rrbeYPn06BQUFfP3rX2+fFNHS0sKXvvQlgGlaANkzCvwi0icWLVrExo0bg8r37NnDpk2byMvLay/bvn0769evZ9u2bWzcuJGvfe1rtLW1AbB06VLWrFlDTU0NNTU17a+5du1ahg8fDlCFbwex1X3/rlKDAr+I9InrrruOESNGBJV/4xvf4NFHH22f1gzw0ksvMX/+fPr168dFF11EQUEBW7ZsobGxkWPHjjFz5kzMjK985Sv88pe/bD9n4cKF/pf4OVBsgS8qnVLg70R+6cvkl74c78tIOKqTzqluzurs96eiooJx48Zx+eWXdyhPpwWQiRBbNLgbIN7/GSKprKmpiRUrVvDb3/426LlYLIDEt80khYWFaT9lWi1+6ZGDrzwWNGD3rW99i0mTJnHZZZdx8803c/To0fbnIh2wKygoAJikAbvUsWvXLmpra7n88svJz8+nrq6Oq666ir1792oBZIwp8JMYt17JYvD02UEDdiUlJVRVVfHee+9x6aWXsmrVKqB3A3Y7d+4E2IcG7FLG9OnT2b9/P7t372b37t3k5uby9ttvM2bMGC2AjDEFfumR/uOnBQ3YzZkzh8xMX69hUVFRewstCgN2R9CAXdI6UPFo0ALIzmgBZGypj78b/juB3WWfj/OVJId169b551ZTX19PUVFR+3P+gbmsrKywBuw8/gG7g5zDzJYAS4AOUwMlMYy68dtd/t7s3r27w/daABk7avFL1KxYsYLMzEzuvPNOQBlLRRKVAr9ERXl5ORs2bOC5555rD+K9HbDzaMBOJMoU+KXXNm7cyOrVq6moqGDgwIHt5VEYsBuOBuxEoi6t+/g1k6fnDlQ8yszyHRw8eJDc3FyWL1/OqlWraGlpoaSkBPAN8P7oRz/qMGCXmZkZNGC3aNEimpubmTdvXocBu7vuuss/nXMMGrATibq0DvwSPv8fyVADdosXL+70vEgH7ADM7APn3Ee9uW4RCRZWV4+Z7Taz983sz2a21SsbYWabzKzG+3d4wPGPmNlOM6s2s+sDyq/2XmenmT2uaXoiIrHXkz7+zzrnrnDOFXrflwKVzrkJQKX3PWY2BZgPTAXmAj80swzvnKfwTb+b4H3N7f1biIy6eSSa7rnnHvY8cadSEEtS6M3g7k2AfxSuHPhiQPl651yLc64W2Alca2Y5wFDn3BveYN3PAs6RJKKVzsEWLVpE9m3Lg8qVglgSUbiB3wG/NbO3vEUzAKOdc40A3r/ZXvk4IHA+Xp1XNs57fG55kETKpCcSjuuuu46MAUOCypWCWBJRuIH/0865q4B5wP1mdl0Xx4b6MLouyoMLE3Bhjlq50lNKQSyJKqxZPc65Bu/f/Wb2C+BaYJ+Z5TjnGr1unP3e4XVA4Jr7XKDBK88NUS6ScpSCWBJZty1+MxtkZkP8j4E5+PoZKwD/vedC4CXvcQUw38z6mdlF+AZxt3jdQcfNrMi7Rf1KwDkiKUUpiCWRhdPVMxr4LzN7F9gCvOyc2wiUASVmVgOUeN/jnNsGvABsBzYC9zvn2rzXWgo8jW/Adxfw6yi+F5GYCKfbTymIJZF129XjLaC5PET5IaC4k3NWACtClG/Fl0JVJKUsWLCAvb/6DftajrevaO5sYVtvVjRzNgXx/Ji8MUlJWrkrEgXPP/88b5S+3GkaYqUglkSSdoFfM3NEJN0pO2cP6Q+HiCQ7BX4RkTSjwC8ikmYU+EVE0owCv4j0iYOvPEZ2dnaHjKXf+ta3mDRpEpdddhk333wzR48ebX9OGUtjR4FfRPrE4Omz27OL+pWUlFBVVcV7773HpZdeyqpVqwBlLI01BX7pkpLTSaT6j5/GiBEjOpTNmTOHzEzfLPKioqL2FBXKWBpbCvwiEhfr1q1rX5msjKWxpcAvIjG3YsUKMjMzufPOO4HYZCxNtFTv8aTAH6Hvf//7TJ06lWnTprFgwQJOnTrF4cOHKSkpYcKECZSUlHDkyJH247sauAKmaB9iSRfl5eVs2LCB5557rj2IK2NpbCnwR6C+vp7HH3+crVu3UlVVRVtbG+vXr6esrIzi4mJqamooLi6mrKwM6H7gCvgLCbAPsUhf27hxI6tXr6aiooKBAwe2lytjaWwp8EeotbWV5uZmWltbaWpqYuzYsR0GmxYuXNhhEKqrgSvgpPYhllRzoOJRZs6cSXV1Nbm5uaxdu5YHHniA48ePU1JSwhVXXMF9990HdMxYOnfu3KCMpffeey8FBQVccsklHTKWHjp0CM5mLC2NyxtNQmmTpC2aM1M+/cSfOTlxHnl5eQwYMIA5c+YwZ84c9u3bR05ODgA5OTns3+/blKy+vp6ioqL28/0DVFlZWeTm5lJdXe1/qst9iIElQIeNu0US1agbvx2UrbSzVNWgjKWxpBZ/BNpOnaCp5k1qa2tpaGjg5MmTPPvss50e35OBKxJ8cOrgK4+x54k7aVj7tfaySMc2ulqUU1BQADBJi3JEok+BPwKndv+ZzAtGM2rUKLKysrjlllt4/fXXGT16NI2NjQA0NjaSnZ0NhD9wRRLsQzx4+myyb1veoSzSsY2uFuXs3LkTYB9alCMSdQr8EcgcOorTDdU0NTXhnKOyspLJkyd3GGwqLy/vMAjV1cAVMChZ9iHuP34aGQOGdCiLdGwjjEU5R9CiHJGoS5s+/mjqN3YiAyd+mquuuorMzEyuvPJKlixZwokTJ7j99ttZu3YteXl5vPjii0D3W+1dc801+fj2If41SbgPcaRjG+eW+88JXMjD2UU5B/v4bYikDbX4IzTsb+/kww8/pKqqimeeeYZ+/fpx4YUXUllZSU1NDZWVlR2Wqy9btoxdu3ZRXV3dPisBfANXwDbn3CXOuQdSaTpaXy/KSaTVmPfccw97nrhTCckkKSjw94Ly2Pj0dmyjs0U5nk4X5STKgDfAokWLgsY+lJBMEpUCv/RapGMbYSzKGU6SLMq57rrrgsY+lJBMElXK9/GrRR5dByoepeXj92lrPkZubi7Lly+ntLQ0orGNRYsW0dzczLx58zosyrnrrrv80znHkCKLctatW+fvqun12IdzrtXMOh370JoP6U7KB36JrlE3frv9ceDinMrKypDHR7ooB8DMPnDOfdTba463eCQkA9YAFBYWJvzdksSeAr9IH/InJKusrFRCMkkY6uMX6SNKSCaJOgFEgV8ilqgf6nhYsGABe5/5phKSSVJQV4+EpIDeM88//zxvlL7cYdxDCckkUanFL9ID+oMoqSDswG9mGWb2jplt8L4fYWabzKzG+3d4wLGPeDtKVZvZ9QHlV5vZ+9ptSkQkfnrS4n8Q+CDg+1Kg0jk3Aaj0vsfMpgDzgan4dpP6oZlleOc8hW9+sXabEhGJk7ACv5nlAp8Hng4ovgnwTzMo5+zOUTcB651zLc65WnzJx641sxxgqHPuDe02JSISP+G2+B8Dvg2cCSgb7ZxrBPD+zfbKxwGByVb8u0qN8x6fWx4kkZJviYikmm4Dv5l9AdjvnHsrzNcM1W/vuigPLkyg5FsiIqkmnOmcnwZuNLMbgP7AUDN7FthnZjnOuUavG2e/d3wdEJhQ3b+rVJ33+NxyERGJoW5b/M65R5xzuc65fHyDtq86574MVAD+dIELObtzVAUw38z6mdlF+AZxt3jdQcfNrChZdpsSEUlFvZnHXwaUmFkNUOJ9j3NuG/ACsB3YCNzvnGvzzlmKb4B4J7CLJNxtSkTCc/CVx8jOzu6wOc3hw4cpKSlhwoQJlJSUcOTIkfbntDlN7PQo8DvnXnPOfcF7fMg5V+ycm+D9ezjguBXejlITnXO/Dijf6pybloq7TYlIR4Onz27fSMavrKyM4uJiampqKC4upqysDNDmNLGW0it3Y7XKMtVy1qTSe5H46T9+WoftR6HjhjILFy7ssNGMNqeJnZQO/CKSWPbt20dOTg4AOTk57N/vmxMSuNEMnN2Epr6+PuzNaQD/5jRB4j1FvKvGVDwaWgr8IhJ3sdicRlPEz1LgF5E+8zerX2XHvuPt348ePZrGxkYAGhsbyc72rfvU5jSxpcAvIjETuKFMeXl5h41mtDlN7Cgfv4j0iQMVj9Ly8fu0NR8jNzeX5cuXU1payu23387atWvJy8tr3185cHOazMzMoM1pFi1aRHNzM/PmzeuwOc1dd90FZzenmR+XN5qEFPhFpE+MuvHb7Y8DN6iprKwMebw2p4kdBX5pp2mckVG9SbJRH7+ISJpR4BeJgoOvPMaeJ+5UegJJCgr8IlEwePpssm9b3qFM6QkkUSnwS9R8//vfZ+rUqUybNo0FCxZw6tSpXrV68bVuk2Jv5v7jp5ExYEiHMqUnkESlwC9RUV9fz+OPP87WrVupqqqira2N9evX96rVi691m7R7M6dregJJfAr8EjWtra00NzfT2tpKU1MTY8eO7VWr15OwezNHmpxP6Qkk3jSdU6Ji3LhxfPOb3yQvL48BAwYwZ84c5syZ02Wrt6ioqP18f+s2KyurQ6uXbvZmBpYA5OXl9cn78oskwPvTE+Tk5Cg9gSQUtfglKo4cOcJLL71EbW0tDQ0NnDx5kmeffbbT43vS6iVJW7ZKTyCJSoE/QmdOneDAL1ZS/+P7qP/xfbTUfxDxQCYwxcx2JstAZii/+93vuOiiixg1ahRZWVnccsstvP76671OykWS7M18oOJR9j7zTaqrq8nNzWXt2rWUlpayadMmJkyYwKZNmygtLQU6pieYO3duUHqCe++9l4KCAi655JIO6QkOHToEZ9MTlMbljUpKSMmunlispDxcuYb+F1/NqJv/Adf2Ce6TFi6a91UeubmY0tJSysrKKCsrY/Xq1R0GMhsaGpg9ezY7duwgIyODpUuXAvwFmAK8gm8gM+m2pMzLy2Pz5s00NTUxYMAAKisrKSwsZNCgQZSXl1NaWhrU6r3jjjt4+OGHaWhoaG/1ZmRktLd6PV8BnojX+wqXPz1BYGoCUHoCSUwpGfj72pmWJk7t2caFN3wDAMvIwjKyaNr5JgsX/ivgG8icNWsWq1ev7nQgMz8/n2PHjgGcdM45M/MPZMY08EfjD+WMGTO49dZbueqqq8jMzOTKK69kyZIlnDhxIuKkXPhat0+ThH8IRRKZAn8EWo/uJWPgUA698hin99fSb0wBw4uX0HbyaEQDmdXV1f6nOh3ITAbLly9n+fKOi5j69esXcavXa90+0CcXK31CeYuSg/r4I+DOtHF67y6GXHkDY+9+HMvqx7HNL3Z+fBQGMjU3Ozmk2v7L4UjH95zsFPgjkDlkJBlDRtJv7EQABk78NKf37SJj0LA+G8hM5Bks+sUXSS4K/BHIGDyczKEj+eSQL2if+su7ZI3MY2DBjIim7wGDvNk8XwFeisubEpG0oT7+CI2YfR8HN3wX19ZK5rAxXHjDQ+DOsGnT0z0eyLzmmmvygZ34BjE1kCkifUqBP0Lnj76YnIWPBZVHMpAJbHPOFUb5EkVEQlJXj4hImlHgFxFJMwr8IhJz0dy7gRRIeRJrCvwiElPR3rsBX8qTCSTx3g2xpsAvIjEXzb0b8FKekMB7NySabgO/mfU3sy1m9q6ZbTOz5V75CDPbZGY13r/DA855xLv1qjaz6wPKrzaz93VbJpK+AvduyMnJ4YILLuh274Zwdiyjm70b4r3yPZEWOobT4m8BPuecuxy4AphrZkX40sJWOucmAJXe95jZFGA+MBXfbdcPzSzDe62n8G2codsykTSlvRvir9vA73xOeN9meV8OuAnw7wxRztlbrJuA9c65FudcLb6FSdeaWQ4w1Dn3hm7LRNJXuu/dkAjC6uM3swwz+zOwH9jknHsTGO2cawTw/s32Dh8H7Ak43X/7Nc57fG55qJ8X99syEekbgXs3OOeorKxk8uTJEe9YhlKe9FhYK3edc23AFWY2DPiFmU3r4vBQ/faui/JQP28NsAagsLBQ28uJpJBo792glCc916OUDc65o2b2Gr6++X1mluOca/S6cfZ7h9UB4wNO899+1XmPzy0XkTQTzb0bUMqTHgtnVs8or6WPmQ0AZgMfAhXAQu+whZy9xaoA5ptZPzO7CN8g7havO+i4mRXptkzSiRYrSaIJp48/B/i9mb0H/AlfH/8GoAwoMbMaoMT7HufcNuAFYDuwEbjf6yoCWIpvK72dwC5S8LYskaZsSfxpsZIkom67epxz7wFXhig/BBR3cs4KYEWI8q349lEVSRv+xUpZWVnti5VWrVrFa6+9BiTX/sySGrRyV6QPpdNiJd3pJo+UCvzqZpFEo8VKkohSKvCLJBotVpJEpMAv0oe0WEkSkbZeFOlCb7sOtVhJEpECv0gf02IlSTTq6klj0R4MP3r0KLfeeiuTJk1i8uTJvPHGGxEvVJo+fTrANC1UklQVz8koCvwSNbl/+9+ZO3cuH374Ie+++y6TJ0+OeKHSmjVrAKrQQiWRqFPgl6g409LEqT3bWLx4MQDnn38+w4YNi3hXpZkzZ/pfWum7RaJMffwSFa1H95IxcCh333037777LldffTU/+MEPulyoVFRU1H6+f6FSVlZWjxYq4dvYh7y8vKi+n2jdgvtfZ3fZ56PyepJ4kvH/WC1+iQp3po3Te3exdOlS3nnnHQYNGtTerRPyeC1UEokbBX6JiswhI8kYMpIZM2YAcOutt/L2229roZJIAlLgl6jIGDyczKEjqa6uBqCyspIpU6ZEvFBp8+bN/pfWQiWRKFMfv0TNiNn3ceedd3L69GkuvvhifvKTn3DmzJmIFiotWrQIfJlcn0YLlUSiSoFfoub80Rez9Sdbg8ojWahUVVWFmVU55x6I+oWKpDl19YiIpBkFfhGRNKPALyKSZhT4RSTmopnXCW1A32MK/H1Eu4GJdO7BBx+MWl4ntAF9jynw94I700bDT77O/p/7Uu62NR9n3/p/pH7NV9m3/h9pO3Wi/Vi1WCTVRNq4OXbsGH/4wx+iltcJbwN6lNcpbCkR+OPVuj6+tYKsC89ujH1s84v0z7+ccUt+TP/8yzm22TdnXS0WkbM++ugjRo0axd13382VV17Jvffey8mTJ1NyA/pElRKBPx5ajx2k+aM/MfjyOe1lTTvfZNC0YgAGTSumqca3+lQtFkkV0Whgtba28vbbbyuvUxwp8EfoSOUahs26h8CembaTR8kcPAKAzMEjOHPyKBCdFotIsvPfmefm5pKbm6u8TnGkwB+Bpp1bOG/QMPqNKQjr+Gi0WHSrmrzOnDqhGSwBxowZw/jx46OW1wltQN9jStkQgZb67TTXvEndrq24ttO4lmYO/uq7ZAwaRuuJw2QOHkHricOcN2gYEJ0Wi3NuDbAGoLCwMOQfB0lMhyvXMPfBBfz85z/n9OnTNDU1sXLlSoqLiyktLaWsrIyysjJWr17dYTyooaGB2bNns2PHDjIyMgLHg6YAr+AbD0rKPEZPPPFE1PI6aQP6nlPgj8Dwzyxi+GcWAXDq4/c4tuUXjPy7b3Lk9+s4WVXJBUW3cbKqkoEFvlvZG2+8kTvuuIOHH36YhoaG9hZLRkZGqBbLE3F6WxJF/r7wUDuTnX/++bz00ku89tprgG8Gy6xZs1i9enWn40H5+fkdxoPMzD8elJSB7oorrmDr1ujkdUIb0PeYunqiaGjRrZza/Q71a77Kqd3vMLToNqBji2Xu3LlBLRYgH1+LZRdJ+ovsp/ULHQXuTBarGSwi3VGLv5f6511G/7zLAMgYMJTR81eGPE4tlvR0dmeyZ5gxYwYPPvhgn89g6cstKSF621JK/HTb4jez8Wb2ezP7wMy2mdmDXvkIM9tkZjXev8MDznnEG4CqNrPrA8qvNrP3k31wSiRc8diZTFMXpTvhdPW0Av/TOTcZKALuN7MpQClQ6ZybAFR63+M9Nx+Yim/w6YdmluG91lP4WiJarCRpIdo7k6EZLBIF3Xb1OOcagUbv8XEz+wBf3+JNwCzvsHLgNeA7Xvl651wLUGtmO4FrzWw3MNQ59wZAsg9OiYQrmjuTaQaLREOP+vjNLB+4EngTGO39UcA512hm2d5h44DNAaf5B6E+8R6fWy6S0qK5MxkaD5IoCHtWj5kNBv4DeMg5d6yrQ0OUuS7KQ/0sLVYSkaSTLLPawgr8ZpaFL+g/55z7T694n5nleM/nAPu98jpgfMDp/kGoOu/xueVBNDglItJ3wpnVY8Ba4APn3L8EPFUBLPQeL+TsQFMFMMNmRNIAAAvtSURBVN/M+pnZRfgGcbd43ULHzaxIg1MiIvETTh//p4G7gPfN7M9e2T8AZcALZrYY+Bi4DcA5t83MXgC245sRdL9zrs07bynwU2AAGpwSEQHOro3YXfb5mPy8cGb1/Beh++cBijs5ZwWwIkT5VmBaTy5QJBXkl74cs19qke4oZYOISJpR4E9TfTXzwL8d5Re+8AWAiNMPT58+HWCaVniLRJ8Cv0TVudtRRrqB9po1awCq0ArvhNDbaYrJMMUxnSjwS9SE2o4y0g20Z86c6X8JbUcpEmUK/BI1obajVPphkcSjwC9Roe0oRZKH8vFLVJy7HeWrZ1r48pe/3J5+OCcnp0/SD6PtKEV6LOlb/Ik+aJQsuTt6a/hnFpF7fzm5S9cx6sZv87nPfY5nn3024vTDmze35/nTCm+RKFOLX/pUaWlpROmHFy1aBL7Ffk+jFd4iUZX0LX5JPP3zLmPDhg0AXHjhhVRWVlJTU0NlZSUjRoxoP27ZsmXs2rWL6upq5s2b115eWFhIVVUVQJVz7gHXSce/JLdPfbuC80df0us1H8AU7erXMwr8IhIX0VrzAfwF7erXIwr8IhJzdXV1UVvzAZz07gq15iNM6uMXkZh76KGHGDbrHtzppvayrtZ8FBUVtR/nX/ORlZVFbm5u+37GJNCaj0Sf0KEWv4jE1IYNG8jOzk65NR+JHuwDKfCL9DElruvoj3/8IxUVFdQ9dQ8HKh7l1Vdf7bDmA+iTNR/a1e8sBX6RPqbEdR2tWrWKurq6qK35AAZpV7+eUeAX6UNKXBe+0tJSNm3axIQJE9i0aROlpaVAxzUfc+fODVrzAeQDO4FdaM1HWDS4K9KH/InrojGIGaDLQUwzWwIsAcjLy4vem+kD/fMuY0PZI8DZNR+hLFu2jGXLlgWVFxYWAmxzzhX24WWmHLX4RfpIPBLXea+j/mzpklr8In0kHonrRMKhFr9IH1HiOklUavGLxJgS10m8KfCLxEA0BjGrqqowsyrn3AN9erGS8tTVIyKSZhT4RUTSjAK/iEiaUR9/BFqPHeDgy/9C24kjmJ3H4CuuZ2jhTbQ1H+fgS6tpPbaPzKGjGfnFUjL6DwZ8y9TXrl1LRkYGjz/+ONdffz3QcSMJ4BXgQW08IiJ9SYE/EudlMPyzi+k3poAzLU00lj9E//wrOfn+7+iffzkXFN3GXze/yLHNLzJ81t2cPvgx63/ty8HS0NDA7Nmz2bFjBxkZGYEbSUzBF/jnohkbcZVMWRYl9vyfj91ln0+q1w6krp4IZA4e0b4a87x+A8m6cDxtxw/RtPNNBk0rBmDQtGKaanzzrptrNmsjCRFJGGrx91LrX/dxet9H9Bs7kbaTR8kc7NtTNnPwCM6cPApA24lDjB9/NjtjJBtJRCv/Sqxas7FquYhIz3Xb4jezdWa238yqAspGmNkmM6vx/h0e8Nwj3sbH1WZ2fUD51Wb2fiptinzmdDMHfrGSEcVf5bx+Azs9LlSPfU9zsCj/iohESzhdPT8lOPd3KVDpnJsAVHrfY2ZTgPnAVO+cH5pZhnfOU/harCmxKbJra+XAL1YyaMosBk78bwBkDBpG64nDALSeOMx5g4YBkDnkQuVgEZGE0W3gd879ATh8TvFNQLn3uJyz/dI3Aeudcy3OuVp8ObKvNbMcYKhz7o1U6Mt2znHo1z8g68LxDL325vbygQUzOFnlW5F5sqqSgQUzABhQMEMbSYhIwoi0j3+0c64RwDnXaGbZXvk4YHPAcf4+60+8x+eWh5To+cRb6rdzctvvyRqVT8NP/h6A4dd9haFFt3LwpTJOvPdbMoeOYuRNviX654/6FB9fcHmnOViuueaafHx/JH+NZvRIAtEMp9QU7cHdUP32rovykJxza4A1AIWFhSGPi+cHsn/uVD71nQ0hnxs9f2XI8gv+25fYVfazoHJtJJE+NOAtiSLS6Zz7vO4bvH/3e+V1wPiA4/x91nXe43PLJUW0HjvA3ucfof7H99Hw9Nc4ttXXY5XuG4uLJKJIA38FsNB7vJCz/dIVwHwz62dmF+EbxN3idQsdN7Mi9WWnKG9R27iv/ogxd32X42+/zOmDH6f9xuIiiSic6ZzPA28AE82szswWA2VAiZnVACXe9zjntgEvANuBjcD9zrk276WW4ssjrk2RU1Bni9q0sbhI4um2j985t6CTp4o7OX4FsCJE+VZ8m0hIigtc1LbvN323sXiiTwIQSVRK2SBRFf6itt5vLK5Fbclpz549fPazn20fD/rBD34ARD4ehJfkUONB4VPgl6gJtajNv7E4kPAbi+eXvqzpizGQmZnJ9773vfbxoCeffJLt27dHPB6EL8lhSiwMjRUFfomKzha1pfvG4tGe7UQKtG5zcnK46qqrAN940OTJk6mvr494PAglOewxJWmTqOhsUVvpj9J8Y/EQKby3b9/OT3/6U4qLiyktLaWsrIyysjJWr17doXWbDim8W/+6j3feeYcZM2awb19k40GxSHIYag1GX94d9vWaDwV+iYrOFrWl+8bimYNHtGds9c928rduX3vtNcDXup01axarV6/utHWbn5/foXVrZv7WbdIGfv940PM/fIyhQ4d2ely0xoPoZlFoOlFXj0iM+Gc7dde6DZXCu76+vkezncxsq5ltPXDgQF+9nV755JNP2seDbrnlFiC5xoOSnQK/SAwEznaKRes2kWc7OedYvHhx+3iQv1sj0vEglOSwx9TVI9LHuprtlJOTk3at2z/+8Y8888wzHcaDXrnOKC2NbDxISQ57ToE/TWiaYnx0N9uptLQ0qHV7xx138PDDD9PQ0NDeus3IyAjVun0iHu+pt/7mb/4G51yHz+QNN9wAENF4EEpy2GMK/CJ9KNRsJ7VuJd4U+EX6UKjZTmrdSrxpcFdEJM2oxS8iMaXxpvhTi19EJM0o8IuIpBkFfhGRNJN0ffzJ3D+ozbZFJBGoxS8ikmaSrsUvIhIvse5xCPx50ewpUIs/xWlXKekpfV5SnwK/9Cn94RFJPAr8IiJpRoFfRCTNKPCLiKQZBX4RkTSjwJ/CNKgqIqEo8IuIpBkFfpEY0xRXiTcFfhGRNKPALyKSZmIe+M1srplVm9lOMyuN9c9PUEOjWSep0JWwceNGgGn6nASJ6mclRahOeiimgd/MMoAngXnAFGCBmU0J59xUCGZ+ge+lra0NII8I6iSZ9OT/rq2tjfvvvx9gBylcJz2VLp+VnlCdRCbWLf5rgZ3OuY+cc6eB9cBNMb6GhLJlyxaAFtXJWVu2bKGgoADgdCzqJFkaFan8WYn0/yCV66QvxTot8zhgT8D3dcCMcw8ysyXAEu/bE2ZWDYwEDvb5FcaQrWYk0AaMDSjuSZ1AEtWLrQ7rMH+dDAU+5ZWlZJ2EWR9w9v0MJ/LPSo/qpAfXFlU9/Lm9/f1Jis+JXw9+f/zv6VOdHRTrwG8hylxQgXNrgDUdTjTb6pwr7KsLiwcz2wqsBq4/56mw6sT/GqlUL4F14py7N+CptK4T51yhmd1GhJ+VVKsT6P3vT6rWSTjvKdZdPXXA+IDvc4GGGF9DolGdBFOdhKZ6CaY6iUCsA/+fgAlmdpGZnQ/MBypifA2JRnUSTHUSmuolmOokAjHt6nHOtZrZA8BvgAxgnXNuW5inB93Sp4A1vawTSL16UZ0EWwP6/Qmht5+VlKyTcA4y54K6w0REJIVp5a6ISJpR4BcRSTNJEfhTLc2Dma0zs/1mVtWL11CdBL+G6iT066hegl8jvevEOZfQX/gGbHYBFwPnA+8CU+J9Xb18T9cBVwFVqhPVSV/ViepFddLZVzK0+FMuzYNz7g/A4V68hOokmOokNNVLsLSvk2QI/KHSPIyL07UkCtVJMNVJaKqXYGlfJ8kQ+MNK85BmVCfBVCehqV6CpX2dJEPg15LsYKqTYKqT0FQvwdK+TpIh8GtJdjDVSTDVSWiql2BpXycJH/idc62Af0n2B8ALrmfL9xOOmT0PvAFMNLM6M1vck/NVJ8FUJ6GpXoKpTpSyQUQk7SR8i19ERKJLgV9EJM0o8IuIpBkFfhGRNKPALyKSZhT4RUTSjAK/iEia+f/NNqX74AsUjAAAAABJRU5ErkJggg==\n"
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.random.randn(1000,100) # 1000个数据\n",
    "node_num = 100 # 各隐藏层的节点数\n",
    "hidden_layer_size = 5 # 隐藏层数\n",
    "activations = {} # 激活值的结果保存在这里\n",
    "\n",
    "for i in range(hidden_layer_size):\n",
    "    if i!=0:\n",
    "        x = activations[i-1]\n",
    "    w = np.random.randn(node_num,node_num)/np.sqrt(node_num)\n",
    "\n",
    "    z = np.dot(x,w)\n",
    "    a = sigmoid(z) # sigmoid函数\n",
    "    activations[i] = a\n",
    "\n",
    "# 绘制直方图\n",
    "for i ,a in activations.items():\n",
    "    plt.subplot(1,len(activations),i+1)\n",
    "    plt.title(str(i+1)+'-layer')\n",
    "    plt.hist(a.flatten(),30,range=(0,1))\n",
    "plt.show()"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-10-02T07:28:48.310094300Z",
     "start_time": "2023-10-02T07:28:47.283096100Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "<p>我们需要确保激活函数的输出值分散比较广，这样才有机会进行高效的学习</p>"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "## 6.4.正则化"
   ],
   "metadata": {
    "collapsed": false
   }
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
